2024-03-18T22:20:06.222 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T22:20:06.223 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174 -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/orig.config.yaml 2024-03-18T22:20:06.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T22:20:06.368 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174 branch: wip-batrick-testing-20240318.181317 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7609174' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240318.181317 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240318.181317 sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3005 sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 sleep_before_teardown: 0 subset: 433/512 suite: fs suite_branch: wip-batrick-testing-20240318.181317 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjAIo9xmZIWm7Zk56eF9gU6weQ4WnYLWAksOyYFegQ9GOc+kHgYa3cMirSiJqUKAcC6s0uwpGZLHe4uzP6aw6I= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2pEfpV+KB6JATOsaf54zwAI0UR1KbypFXMSkYc2jTtEILFQUobg01tWfQaftT5QDAeF7qG+zktm12SxLj3ZqY= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgz/8U9BkJeimhJd/RJL/4J/oJQnxOuL5NVIxKukK8hBmUODGHPxMOdSalionouFaFs1RfWeW3Tcsc621V7XB8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_21:05:49 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T22:20:06.368 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa; will attempt to use it 2024-03-18T22:20:06.369 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks 2024-03-18T22:20:06.369 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T22:20:06.371 INFO:teuthology.task.internal:Checking packages... 2024-03-18T22:20:06.392 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b6ae24918b03b1b8a19c9263a50655bba80bb810' 2024-03-18T22:20:06.392 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T22:20:06.392 INFO:teuthology.packaging:ref: None 2024-03-18T22:20:06.393 INFO:teuthology.packaging:tag: None 2024-03-18T22:20:06.393 INFO:teuthology.packaging:branch: wip-batrick-testing-20240318.181317 2024-03-18T22:20:06.393 INFO:teuthology.packaging:sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:20:06.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240318.181317 2024-03-18T22:20:06.538 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2206.gb6ae2491 2024-03-18T22:20:06.539 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T22:20:06.548 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T22:20:06.548 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T22:20:06.558 INFO:teuthology.task.internal:Saving configuration 2024-03-18T22:20:06.578 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T22:20:06.584 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T22:20:06.607 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.066595', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjAIo9xmZIWm7Zk56eF9gU6weQ4WnYLWAksOyYFegQ9GOc+kHgYa3cMirSiJqUKAcC6s0uwpGZLHe4uzP6aw6I='} 2024-03-18T22:20:06.627 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.064872', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2pEfpV+KB6JATOsaf54zwAI0UR1KbypFXMSkYc2jTtEILFQUobg01tWfQaftT5QDAeF7qG+zktm12SxLj3ZqY='} 2024-03-18T22:20:06.650 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.062896', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgz/8U9BkJeimhJd/RJL/4J/oJQnxOuL5NVIxKukK8hBmUODGHPxMOdSalionouFaFs1RfWeW3Tcsc621V7XB8='} 2024-03-18T22:20:06.650 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T22:20:06.656 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-18T22:20:06.656 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-18T22:20:06.656 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-18T22:20:06.656 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T22:20:06.758 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f44979fa160>, signals=[15]) 2024-03-18T22:20:06.758 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T22:20:06.765 INFO:teuthology.task.internal:Opening connections... 2024-03-18T22:20:06.765 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T22:20:06.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:20:06.858 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-18T22:20:06.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:20:06.945 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-03-18T22:20:06.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:20:07.033 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T22:20:07.040 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-03-18T22:20:07.058 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-03-18T22:20:07.058 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-03-18T22:20:07.115 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T22:20:07.116 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T22:20:07.116 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-03-18T22:20:07.137 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-18T22:20:07.155 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-18T22:20:07.155 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-18T22:20:07.212 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-18T22:20:07.212 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-18T22:20:07.212 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-18T22:20:07.212 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-18T22:20:07.212 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T22:20:07.213 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T22:20:07.213 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-18T22:20:07.237 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-03-18T22:20:07.254 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-03-18T22:20:07.255 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-18T22:20:07.311 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-03-18T22:20:07.311 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-03-18T22:20:07.311 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-03-18T22:20:07.311 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-03-18T22:20:07.311 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T22:20:07.312 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T22:20:07.312 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-18T22:20:07.336 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T22:20:07.347 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T22:20:07.352 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T22:20:07.352 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-03-18T22:20:07.354 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-18T22:20:07.357 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-03-18T22:20:07.373 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T22:20:07.378 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T22:20:07.378 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-03-18T22:20:07.411 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-18T22:20:07.413 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-03-18T22:20:07.651 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T22:20:07.727 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T22:20:07.740 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-18T22:20:07.741 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-18T22:20:07.741 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-18T22:20:07.741 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T22:20:07.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:20:07.758 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-18T22:20:07.815 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-361.el9.x86_64 2024-03-18T22:20:07.815 INFO:teuthology.task.kernel:Running kernel on smithi132: 5.14.0-361.el9.x86_64 2024-03-18T22:20:07.815 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum install -y kernel 2024-03-18T22:20:10.104 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-18T22:20:11.679 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-18T22:20:13.745 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-03-18T22:20:32.136 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-18T22:20:37.394 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-03-18T22:20:38.979 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-03-18T22:20:40.266 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-18T22:20:45.130 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-18T22:20:47.195 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T22:20:47.196 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T22:20:47.334 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-18T22:20:47.341 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-18T22:20:47.341 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:Install 4 Packages 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:20:47.342 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 76 M 2024-03-18T22:20:47.343 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 123 M 2024-03-18T22:20:47.343 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-18T22:20:47.498 INFO:teuthology.orchestra.run.smithi132.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 387 kB/s | 44 kB 00:00 2024-03-18T22:20:50.307 INFO:teuthology.orchestra.run.smithi132.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 5.1 MB/s | 15 MB 00:02 2024-03-18T22:20:51.499 INFO:teuthology.orchestra.run.smithi132.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 6.9 MB/s | 27 MB 00:04 2024-03-18T22:20:52.016 INFO:teuthology.orchestra.run.smithi132.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 7.2 MB/s | 33 MB 00:04 2024-03-18T22:20:52.016 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:20:52.016 INFO:teuthology.orchestra.run.smithi132.stdout:Total 16 MB/s | 76 MB 00:04 2024-03-18T22:20:52.685 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-18T22:20:53.037 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-18T22:20:53.037 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-18T22:20:54.036 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-18T22:20:54.036 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-18T22:20:56.489 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-18T22:20:57.110 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-18T22:20:57.200 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T22:20:59.930 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T22:21:00.072 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T22:21:04.752 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T22:21:04.797 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:09.650 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:38.965 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:38.966 INFO:teuthology.orchestra.run.smithi132.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T22:21:38.966 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:21:39.009 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:39.344 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:39.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-18T22:21:39.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-18T22:21:39.345 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-18T22:21:39.781 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-18T22:21:39.782 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:21:39.782 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-18T22:21:40.096 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T22:21:40.188 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T22:21:40.189 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T22:21:40.189 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-18T22:21:40.189 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T22:21:40.189 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T22:21:40.189 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-18T22:21:40.190 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T22:21:40.190 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T22:21:40.269 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-18T22:21:40.269 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-18T22:21:40.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-03-18T22:21:40.302 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-03-18T22:21:40.303 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:21:40.304 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T22:21:40.305 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T22:21:40.305 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 5.14.0-430.el9.x86_64 2024-03-18T22:21:40.305 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T22:21:40.919 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-03-18T22:21:42.318 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T22:21:42.355 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-03-18T22:21:42.357 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-03-18T22:21:42.372 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.SIRKjaycKL 2024-03-18T22:21:42.372 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SIRKjaycKL 2024-03-18T22:21:42.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.SIRKjaycKL 2024-03-18T22:21:42.585 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.SIRKjaycKL is 6KB 2024-03-18T22:21:42.634 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.SIRKjaycKL 2024-03-18T22:21:42.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T22:21:42.717 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T22:21:42.717 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T22:21:42.717 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T22:21:42.717 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-18T22:21:42.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-18T22:21:42.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-03-18T22:22:12.858 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-18T22:22:12.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:22:12.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:22:31.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-03-18T22:22:40.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:22:40.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:23:40.203 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T22:23:52.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:23:52.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:23:52.696 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-18T22:23:53.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:23:53.114 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-18T22:23:53.114 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-03-18T22:23:53.129 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-430.el9.x86_64 2024-03-18T22:23:53.129 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-18T22:23:53.129 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T22:23:53.129 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T22:23:54.130 DEBUG:teuthology.parallel:result is None 2024-03-18T22:23:54.131 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T22:23:54.138 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T22:23:54.138 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T22:23:54.141 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T22:23:54.144 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T22:23:54.161 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T22:23:54.168 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T22:23:54.178 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T22:23:54.178 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T22:23:54.198 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T22:23:54.201 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T22:23:54.464 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T22:23:54.472 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T22:23:54.472 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T22:23:54.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:23:54.488 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T22:23:54.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:23:54.505 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T22:23:54.507 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T22:23:54.531 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T22:23:54.575 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.590 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.603 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.614 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.626 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T22:23:54.627 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T22:23:54.635 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T22:23:54.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T22:23:54.640 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T22:23:54.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T22:23:54.671 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T22:23:54.680 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T22:23:54.680 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T22:23:54.711 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T22:23:54.713 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T22:23:54.728 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:54.809 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:54.861 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:54.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:54.988 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:23:54.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T22:23:55.057 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:55.099 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:55.155 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:55.239 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:55.283 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:23:55.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T22:23:55.351 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:55.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T22:23:55.449 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:55.529 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T22:23:55.576 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:23:55.576 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T22:23:55.646 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-03-18T22:23:55.648 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-18T22:23:55.651 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-03-18T22:23:55.686 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T22:23:55.703 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T22:23:55.734 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T22:23:56.156 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T22:23:56.163 INFO:teuthology.task.internal:Starting timer... 2024-03-18T22:23:56.163 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T22:23:56.173 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T22:23:56.181 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-18T22:23:56.182 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-03-18T22:23:56.231 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-03-18T22:23:56.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-18T22:23:56.296 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-18T22:23:56.298 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-03-18T22:23:56.349 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-03-18T22:23:56.351 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T22:23:56.351 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-03-18T22:23:56.376 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-03-18T22:23:56.377 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-18T22:23:56.398 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-18T22:23:56.398 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-03-18T22:23:56.425 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-03-18T22:23:56.425 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-03-18T22:23:56.425 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T22:23:56.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:23:56.454 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T22:23:56.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:23:56.482 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T22:23:56.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T22:23:56.510 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T22:23:56.510 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-03-18T22:23:56.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-18T22:23:56.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-03-18T22:23:56.618 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T22:23:56.629 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-18T22:23:56.634 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-18T22:23:56.758 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T22:23:56.772 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T22:23:56.773 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi132.front.sepia.ceph.com,smithi144.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T22:50:17.097 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-03-18T22:50:17.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:50:17.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:50:17.190 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-03-18T22:50:17.272 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-03-18T22:50:17.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-18T22:50:17.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:50:17.367 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-18T22:50:17.437 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-18T22:50:17.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-18T22:50:17.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T22:50:17.529 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-18T22:50:17.600 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-18T22:50:17.601 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T22:50:17.613 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T22:50:17.613 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:50:17.614 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:50:17.616 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:50:17.616 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:50:17.618 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T22:50:17.618 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T22:50:17.646 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:50:17.648 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:50:17.662 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:50:17.665 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:50:17.671 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T22:50:17.687 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T22:50:17.691 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-03-18T22:50:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-18T22:50:17.704 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-03-18T22:50:17.709 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-18T22:50:17.716 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-03-18T22:50:17.721 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:50:17.725 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:50:17.730 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-03-18T22:50:17.737 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:50:17.742 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:50:17.746 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T22:50:17.762 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T22:50:17.786 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:50:17.788 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:50:17.789 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:50:17.789 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-18T22:50:17.789 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.789 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.789 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.790 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.806 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-18T22:50:17.809 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T22:50:17.809 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-18T22:50:17.810 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.810 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.810 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.810 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T22:50:17.811 INFO:teuthology.run_tasks:Running task install... 2024-03-18T22:50:17.819 DEBUG:teuthology.task.install:project ceph 2024-03-18T22:50:17.819 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'}} 2024-03-18T22:50:17.819 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'} 2024-03-18T22:50:17.819 INFO:teuthology.task.install:Using flavor: default 2024-03-18T22:50:17.827 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T22:50:17.827 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-18T22:50:17.828 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:50:17.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:50:17.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:50:17.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:50:17.832 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-18T22:50:17.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:50:17.972 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:50:17.972 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:50:17.978 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:50:17.978 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:50:18.048 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-18T22:50:18.048 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-18T22:50:18.092 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:50:18.093 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:50:18.093 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:50:18.121 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:50:18.121 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:50:18.149 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:50:18.149 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:50:18.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:50:18.161 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T22:50:18.161 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:50:18.161 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T22:50:18.177 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:50:18.177 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:50:18.188 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T22:50:18.189 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T22:50:18.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:50:18.245 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:50:18.256 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T22:50:18.270 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:50:18.324 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:50:18.335 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-18T22:50:18.336 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T22:50:18.337 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-18T22:50:18.390 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-03-18T22:50:18.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-03-18T22:50:18.402 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-03-18T22:50:18.404 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-03-18T22:50:18.574 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-18T22:50:18.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:50:18.626 INFO:teuthology.orchestra.run.smithi196.stdout:74 files removed 2024-03-18T22:50:18.653 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:50:18.660 INFO:teuthology.orchestra.run.smithi132.stdout:74 files removed 2024-03-18T22:50:18.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T22:50:19.321 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 220 kB/s | 83 kB 00:00 2024-03-18T22:50:19.322 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 250 kB/s | 83 kB 00:00 2024-03-18T22:50:19.367 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-03-18T22:50:19.697 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-18T22:50:19.726 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-18T22:50:19.730 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-18T22:50:20.098 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-18T22:50:20.098 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-18T22:50:20.110 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-03-18T22:50:20.508 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-18T22:50:20.562 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-18T22:50:20.775 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-18T22:50:21.992 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-03-18T22:50:22.063 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-18T22:50:22.367 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-18T22:50:22.874 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-18T22:50:22.878 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-18T22:50:23.142 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-18T22:50:25.154 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-18T22:50:27.829 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-03-18T22:50:30.193 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.1 MB 00:01 2024-03-18T22:50:31.730 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-18T22:50:33.160 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 9.5 kB/s | 16 kB 00:01 2024-03-18T22:50:33.172 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-18T22:50:34.149 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-18T22:50:35.148 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-18T22:50:39.032 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-18T22:50:39.640 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-18T22:50:40.220 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-18T22:50:41.352 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:41.353 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:41.365 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:50:41.366 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:50:41.367 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:50:41.368 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:41.370 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:41.372 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:50:41.373 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:50:41.453 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:50:41.466 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:50:41.467 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:50:41.468 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:50:41.469 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:50:41.470 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:50:41.471 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:50:41.472 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:50:41.473 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:50:41.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:50:41.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:50:41.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:50:41.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:50:41.478 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout:Install 150 Packages 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 24 Packages 2024-03-18T22:50:41.479 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:50:41.480 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 224 M 2024-03-18T22:50:41.480 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-18T22:50:41.875 INFO:teuthology.orchestra.run.smithi144.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-18T22:50:42.208 INFO:teuthology.orchestra.run.smithi144.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 2.6 MB/s | 891 kB 00:00 2024-03-18T22:50:42.275 INFO:teuthology.orchestra.run.smithi144.stdout:(3/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 9.4 MB/s | 5.4 MB 00:00 2024-03-18T22:50:42.317 INFO:teuthology.orchestra.run.smithi144.stdout:(4/174): ceph-immutable-object-cache-19.0.0-220 1.3 MB/s | 145 kB 00:00 2024-03-18T22:50:42.443 INFO:teuthology.orchestra.run.smithi144.stdout:(5/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 14 MB/s | 2.4 MB 00:00 2024-03-18T22:50:42.598 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:42.598 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:42.610 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:50:42.611 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:50:42.612 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:50:42.613 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:42.614 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:42.617 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:50:42.618 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:50:42.697 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-03-18T22:50:42.709 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-18T22:50:42.709 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-03-18T22:50:42.709 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:50:42.710 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:50:42.711 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:50:42.712 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:50:42.713 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:50:42.714 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:50:42.715 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:50:42.716 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:50:42.717 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:50:42.718 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:50:42.719 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:50:42.720 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:50:42.721 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:50:42.722 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:50:42.723 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================ 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout:Install 150 Packages 2024-03-18T22:50:42.724 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 24 Packages 2024-03-18T22:50:42.725 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:50:42.725 INFO:teuthology.orchestra.run.smithi144.stdout:(6/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-18T22:50:42.726 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 224 M 2024-03-18T22:50:42.726 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-03-18T22:50:42.751 INFO:teuthology.orchestra.run.smithi144.stdout:(7/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-03-18T22:50:43.571 INFO:teuthology.orchestra.run.smithi132.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-18T22:50:43.582 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-18T22:50:43.735 INFO:teuthology.orchestra.run.smithi144.stdout:(8/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x86 17 MB/s | 17 MB 00:01 2024-03-18T22:50:43.771 INFO:teuthology.orchestra.run.smithi132.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 4.4 MB/s | 891 kB 00:00 2024-03-18T22:50:43.803 INFO:teuthology.orchestra.run.smithi144.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 371 kB/s | 25 kB 00:00 2024-03-18T22:50:43.847 INFO:teuthology.orchestra.run.smithi132.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-18T22:50:44.328 INFO:teuthology.orchestra.run.smithi144.stdout:(10/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 6.2 MB/s | 9.8 MB 00:01 2024-03-18T22:50:44.412 INFO:teuthology.orchestra.run.smithi144.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.6 MB/s | 221 kB 00:00 2024-03-18T22:50:44.489 INFO:teuthology.orchestra.run.smithi144.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 351 kB/s | 27 kB 00:00 2024-03-18T22:50:44.547 INFO:teuthology.orchestra.run.smithi132.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-03-18T22:50:44.556 INFO:teuthology.orchestra.run.smithi144.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 476 kB/s | 32 kB 00:00 2024-03-18T22:50:44.714 INFO:teuthology.orchestra.run.smithi144.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 4.6 MB/s | 746 kB 00:00 2024-03-18T22:50:44.790 INFO:teuthology.orchestra.run.smithi144.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.1 MB/s | 163 kB 00:00 2024-03-18T22:50:44.797 INFO:teuthology.orchestra.run.smithi132.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-03-18T22:50:44.857 INFO:teuthology.orchestra.run.smithi144.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.9 MB/s | 127 kB 00:00 2024-03-18T22:50:45.024 INFO:teuthology.orchestra.run.smithi144.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 2.9 MB/s | 501 kB 00:00 2024-03-18T22:50:45.233 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-03-18T22:50:45.531 INFO:teuthology.orchestra.run.smithi132.stdout:(6/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 6.4 MB/s | 4.7 MB 00:00 2024-03-18T22:50:45.982 INFO:teuthology.orchestra.run.smithi144.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 5.4 MB/s | 5.1 MB 00:00 2024-03-18T22:50:46.057 INFO:teuthology.orchestra.run.smithi144.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 604 kB/s | 45 kB 00:00 2024-03-18T22:50:46.141 INFO:teuthology.orchestra.run.smithi144.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.5 MB/s | 131 kB 00:00 2024-03-18T22:50:46.225 INFO:teuthology.orchestra.run.smithi144.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.9 MB/s | 162 kB 00:00 2024-03-18T22:50:46.317 INFO:teuthology.orchestra.run.smithi144.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.4 MB/s | 321 kB 00:00 2024-03-18T22:50:46.408 INFO:teuthology.orchestra.run.smithi144.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.2 MB/s | 298 kB 00:00 2024-03-18T22:50:46.651 INFO:teuthology.orchestra.run.smithi144.stdout:(24/174): ceph-common-19.0.0-2206.gb6ae2491.el9 4.2 MB/s | 21 MB 00:04 2024-03-18T22:50:46.676 INFO:teuthology.orchestra.run.smithi144.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 372 kB/s | 99 kB 00:00 2024-03-18T22:50:46.743 INFO:teuthology.orchestra.run.smithi144.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 930 kB/s | 85 kB 00:00 2024-03-18T22:50:46.860 INFO:teuthology.orchestra.run.smithi144.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.4 MB/s | 170 kB 00:00 2024-03-18T22:50:46.960 INFO:teuthology.orchestra.run.smithi144.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 339 kB/s | 34 kB 00:00 2024-03-18T22:50:47.069 INFO:teuthology.orchestra.run.smithi144.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.3 MB/s | 139 kB 00:00 2024-03-18T22:50:47.103 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-18T22:50:47.127 INFO:teuthology.orchestra.run.smithi144.stdout:(30/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 6.8 MB/s | 3.1 MB 00:00 2024-03-18T22:50:47.397 INFO:teuthology.orchestra.run.smithi132.stdout:(7/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-03-18T22:50:47.469 INFO:teuthology.orchestra.run.smithi144.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 4.5 MB/s | 1.8 MB 00:00 2024-03-18T22:50:47.578 INFO:teuthology.orchestra.run.smithi144.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.2 MB/s | 248 kB 00:00 2024-03-18T22:50:47.694 INFO:teuthology.orchestra.run.smithi144.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 423 kB/s | 49 kB 00:00 2024-03-18T22:50:47.803 INFO:teuthology.orchestra.run.smithi144.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 152 kB/s | 16 kB 00:00 2024-03-18T22:50:47.937 INFO:teuthology.orchestra.run.smithi144.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 1.9 MB/s | 264 kB 00:00 2024-03-18T22:50:48.131 INFO:teuthology.orchestra.run.smithi132.stdout:(8/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x86 6.5 MB/s | 17 MB 00:02 2024-03-18T22:50:48.163 INFO:teuthology.orchestra.run.smithi144.stdout:(36/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 3.7 MB/s | 749 kB 00:00 2024-03-18T22:50:48.198 INFO:teuthology.orchestra.run.smithi132.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 379 kB/s | 25 kB 00:00 2024-03-18T22:50:48.221 INFO:teuthology.orchestra.run.smithi144.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 6.7 MB/s | 7.4 MB 00:01 2024-03-18T22:50:48.238 INFO:teuthology.orchestra.run.smithi144.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 182 kB/s | 18 kB 00:00 2024-03-18T22:50:48.388 INFO:teuthology.orchestra.run.smithi144.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.0 MB/s | 920 kB 00:00 2024-03-18T22:50:48.413 INFO:teuthology.orchestra.run.smithi144.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-18T22:50:48.439 INFO:teuthology.orchestra.run.smithi144.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-18T22:50:48.464 INFO:teuthology.orchestra.run.smithi144.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-18T22:50:48.489 INFO:teuthology.orchestra.run.smithi144.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-18T22:50:48.539 INFO:teuthology.orchestra.run.smithi144.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 352 kB/s | 17 kB 00:00 2024-03-18T22:50:48.590 INFO:teuthology.orchestra.run.smithi144.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-18T22:50:48.615 INFO:teuthology.orchestra.run.smithi144.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 135 kB/s | 10 kB 00:00 2024-03-18T22:50:48.640 INFO:teuthology.orchestra.run.smithi144.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-18T22:50:48.666 INFO:teuthology.orchestra.run.smithi144.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-18T22:50:48.691 INFO:teuthology.orchestra.run.smithi144.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-18T22:50:48.724 INFO:teuthology.orchestra.run.smithi144.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-18T22:50:48.750 INFO:teuthology.orchestra.run.smithi144.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 804 kB/s | 46 kB 00:00 2024-03-18T22:50:48.775 INFO:teuthology.orchestra.run.smithi144.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-18T22:50:48.800 INFO:teuthology.orchestra.run.smithi144.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-18T22:50:48.826 INFO:teuthology.orchestra.run.smithi144.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-18T22:50:48.851 INFO:teuthology.orchestra.run.smithi144.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-03-18T22:50:49.310 INFO:teuthology.orchestra.run.smithi144.stdout:(56/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 8.6 MB/s | 48 MB 00:05 2024-03-18T22:50:49.349 INFO:teuthology.orchestra.run.smithi132.stdout:(10/174): ceph-common-19.0.0-2206.gb6ae2491.el9 3.4 MB/s | 21 MB 00:06 2024-03-18T22:50:49.377 INFO:teuthology.orchestra.run.smithi144.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-03-18T22:50:49.419 INFO:teuthology.orchestra.run.smithi144.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 2.7 MB/s | 1.5 MB 00:00 2024-03-18T22:50:49.432 INFO:teuthology.orchestra.run.smithi132.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.6 MB/s | 221 kB 00:00 2024-03-18T22:50:49.444 INFO:teuthology.orchestra.run.smithi144.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 6.1 MB/s | 811 kB 00:00 2024-03-18T22:50:49.469 INFO:teuthology.orchestra.run.smithi144.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-18T22:50:49.482 INFO:teuthology.orchestra.run.smithi132.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 544 kB/s | 27 kB 00:00 2024-03-18T22:50:49.532 INFO:teuthology.orchestra.run.smithi132.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 633 kB/s | 32 kB 00:00 2024-03-18T22:50:49.544 INFO:teuthology.orchestra.run.smithi144.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-18T22:50:49.666 INFO:teuthology.orchestra.run.smithi132.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 5.5 MB/s | 746 kB 00:00 2024-03-18T22:50:49.711 INFO:teuthology.orchestra.run.smithi144.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-18T22:50:49.724 INFO:teuthology.orchestra.run.smithi132.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.7 MB/s | 163 kB 00:00 2024-03-18T22:50:49.736 INFO:teuthology.orchestra.run.smithi144.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 291 kB/s | 78 kB 00:00 2024-03-18T22:50:49.783 INFO:teuthology.orchestra.run.smithi132.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 2.1 MB/s | 127 kB 00:00 2024-03-18T22:50:49.828 INFO:teuthology.orchestra.run.smithi144.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-18T22:50:49.853 INFO:teuthology.orchestra.run.smithi144.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 176 kB/s | 24 kB 00:00 2024-03-18T22:50:49.879 INFO:teuthology.orchestra.run.smithi144.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-18T22:50:49.891 INFO:teuthology.orchestra.run.smithi132.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 4.5 MB/s | 501 kB 00:00 2024-03-18T22:50:49.912 INFO:teuthology.orchestra.run.smithi144.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 964 kB/s | 32 kB 00:00 2024-03-18T22:50:49.937 INFO:teuthology.orchestra.run.smithi144.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-18T22:50:50.029 INFO:teuthology.orchestra.run.smithi144.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-18T22:50:50.055 INFO:teuthology.orchestra.run.smithi144.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-18T22:50:50.080 INFO:teuthology.orchestra.run.smithi144.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-18T22:50:50.105 INFO:teuthology.orchestra.run.smithi144.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 915 kB/s | 69 kB 00:00 2024-03-18T22:50:50.130 INFO:teuthology.orchestra.run.smithi144.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2024-03-18T22:50:50.156 INFO:teuthology.orchestra.run.smithi144.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-18T22:50:50.181 INFO:teuthology.orchestra.run.smithi144.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T22:50:50.206 INFO:teuthology.orchestra.run.smithi144.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 954 kB/s | 47 kB 00:00 2024-03-18T22:50:50.232 INFO:teuthology.orchestra.run.smithi144.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-18T22:50:50.257 INFO:teuthology.orchestra.run.smithi144.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-18T22:50:50.566 INFO:teuthology.orchestra.run.smithi144.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-18T22:50:50.591 INFO:teuthology.orchestra.run.smithi144.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 117 kB/s | 39 kB 00:00 2024-03-18T22:50:50.608 INFO:teuthology.orchestra.run.smithi132.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 7.2 MB/s | 5.1 MB 00:00 2024-03-18T22:50:50.616 INFO:teuthology.orchestra.run.smithi144.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 115 kB/s | 41 kB 00:00 2024-03-18T22:50:50.641 INFO:teuthology.orchestra.run.smithi144.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-18T22:50:50.659 INFO:teuthology.orchestra.run.smithi132.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 906 kB/s | 45 kB 00:00 2024-03-18T22:50:50.666 INFO:teuthology.orchestra.run.smithi144.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T22:50:50.692 INFO:teuthology.orchestra.run.smithi144.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 837 kB/s | 63 kB 00:00 2024-03-18T22:50:50.717 INFO:teuthology.orchestra.run.smithi144.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-18T22:50:50.717 INFO:teuthology.orchestra.run.smithi132.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 2.2 MB/s | 131 kB 00:00 2024-03-18T22:50:50.742 INFO:teuthology.orchestra.run.smithi144.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-18T22:50:50.767 INFO:teuthology.orchestra.run.smithi144.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-18T22:50:50.776 INFO:teuthology.orchestra.run.smithi132.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 2.7 MB/s | 162 kB 00:00 2024-03-18T22:50:50.801 INFO:teuthology.orchestra.run.smithi144.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-18T22:50:50.826 INFO:teuthology.orchestra.run.smithi144.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-18T22:50:50.843 INFO:teuthology.orchestra.run.smithi144.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2024-03-18T22:50:50.868 INFO:teuthology.orchestra.run.smithi144.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-18T22:50:50.876 INFO:teuthology.orchestra.run.smithi132.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.1 MB/s | 321 kB 00:00 2024-03-18T22:50:51.093 INFO:teuthology.orchestra.run.smithi132.stdout:(23/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 2.7 MB/s | 9.8 MB 00:03 2024-03-18T22:50:51.210 INFO:teuthology.orchestra.run.smithi132.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 894 kB/s | 298 kB 00:00 2024-03-18T22:50:51.235 INFO:teuthology.orchestra.run.smithi132.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 698 kB/s | 99 kB 00:00 2024-03-18T22:50:51.261 INFO:teuthology.orchestra.run.smithi132.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-18T22:50:51.320 INFO:teuthology.orchestra.run.smithi132.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-18T22:50:51.370 INFO:teuthology.orchestra.run.smithi132.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 682 kB/s | 34 kB 00:00 2024-03-18T22:50:51.428 INFO:teuthology.orchestra.run.smithi132.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.3 MB/s | 139 kB 00:00 2024-03-18T22:50:51.879 INFO:teuthology.orchestra.run.smithi132.stdout:(30/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 13 MB/s | 48 MB 00:03 2024-03-18T22:50:51.929 INFO:teuthology.orchestra.run.smithi132.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 3.6 MB/s | 1.8 MB 00:00 2024-03-18T22:50:52.013 INFO:teuthology.orchestra.run.smithi132.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.9 MB/s | 248 kB 00:00 2024-03-18T22:50:52.063 INFO:teuthology.orchestra.run.smithi132.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 990 kB/s | 49 kB 00:00 2024-03-18T22:50:52.067 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-18T22:50:52.122 INFO:teuthology.orchestra.run.smithi132.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 281 kB/s | 16 kB 00:00 2024-03-18T22:50:52.205 INFO:teuthology.orchestra.run.smithi132.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 3.1 MB/s | 264 kB 00:00 2024-03-18T22:50:52.339 INFO:teuthology.orchestra.run.smithi132.stdout:(36/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 2.8 MB/s | 3.1 MB 00:01 2024-03-18T22:50:52.431 INFO:teuthology.orchestra.run.smithi132.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-18T22:50:52.465 INFO:teuthology.orchestra.run.smithi132.stdout:(38/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 2.8 MB/s | 749 kB 00:00 2024-03-18T22:50:52.490 INFO:teuthology.orchestra.run.smithi132.stdout:(39/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 121 kB/s | 18 kB 00:00 2024-03-18T22:50:52.648 INFO:teuthology.orchestra.run.smithi132.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 670 kB/s | 106 kB 00:00 2024-03-18T22:50:52.673 INFO:teuthology.orchestra.run.smithi132.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-18T22:50:52.715 INFO:teuthology.orchestra.run.smithi132.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2024-03-18T22:50:52.740 INFO:teuthology.orchestra.run.smithi132.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-18T22:50:52.766 INFO:teuthology.orchestra.run.smithi132.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 346 kB/s | 32 kB 00:00 2024-03-18T22:50:52.791 INFO:teuthology.orchestra.run.smithi132.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-18T22:50:52.858 INFO:teuthology.orchestra.run.smithi132.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-03-18T22:50:52.883 INFO:teuthology.orchestra.run.smithi132.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 86 kB/s | 10 kB 00:00 2024-03-18T22:50:52.908 INFO:teuthology.orchestra.run.smithi132.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-18T22:50:52.934 INFO:teuthology.orchestra.run.smithi132.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 589 kB/s | 44 kB 00:00 2024-03-18T22:50:52.967 INFO:teuthology.orchestra.run.smithi132.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-18T22:50:53.001 INFO:teuthology.orchestra.run.smithi132.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-18T22:50:53.026 INFO:teuthology.orchestra.run.smithi132.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 507 kB/s | 46 kB 00:00 2024-03-18T22:50:53.133 INFO:teuthology.orchestra.run.smithi132.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-03-18T22:50:53.177 INFO:teuthology.orchestra.run.smithi132.stdout:(54/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 282 kB 00:00 2024-03-18T22:50:53.202 INFO:teuthology.orchestra.run.smithi132.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-18T22:50:53.219 INFO:teuthology.orchestra.run.smithi132.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 403 kB/s | 34 kB 00:00 2024-03-18T22:50:53.303 INFO:teuthology.orchestra.run.smithi132.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-03-18T22:50:53.370 INFO:teuthology.orchestra.run.smithi132.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-03-18T22:50:53.403 INFO:teuthology.orchestra.run.smithi132.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.4 MB/s | 811 kB 00:00 2024-03-18T22:50:53.428 INFO:teuthology.orchestra.run.smithi132.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2024-03-18T22:50:53.470 INFO:teuthology.orchestra.run.smithi132.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-18T22:50:53.562 INFO:teuthology.orchestra.run.smithi132.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-18T22:50:53.588 INFO:teuthology.orchestra.run.smithi132.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 494 kB/s | 78 kB 00:00 2024-03-18T22:50:53.672 INFO:teuthology.orchestra.run.smithi132.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-18T22:50:53.689 INFO:teuthology.orchestra.run.smithi132.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 198 kB/s | 24 kB 00:00 2024-03-18T22:50:53.705 INFO:teuthology.orchestra.run.smithi132.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 157 kB/s | 18 kB 00:00 2024-03-18T22:50:53.730 INFO:teuthology.orchestra.run.smithi132.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-18T22:50:53.747 INFO:teuthology.orchestra.run.smithi132.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 765 kB/s | 32 kB 00:00 2024-03-18T22:50:53.831 INFO:teuthology.orchestra.run.smithi132.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-18T22:50:53.865 INFO:teuthology.orchestra.run.smithi132.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-18T22:50:53.890 INFO:teuthology.orchestra.run.smithi132.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-18T22:50:53.915 INFO:teuthology.orchestra.run.smithi132.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 829 kB/s | 69 kB 00:00 2024-03-18T22:50:53.940 INFO:teuthology.orchestra.run.smithi132.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-18T22:50:53.966 INFO:teuthology.orchestra.run.smithi132.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-18T22:50:53.991 INFO:teuthology.orchestra.run.smithi132.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-18T22:50:54.016 INFO:teuthology.orchestra.run.smithi132.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-18T22:50:54.041 INFO:teuthology.orchestra.run.smithi132.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-18T22:50:54.084 INFO:teuthology.orchestra.run.smithi132.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 43 kB 00:00 2024-03-18T22:50:54.234 INFO:teuthology.orchestra.run.smithi132.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 281 kB/s | 41 kB 00:00 2024-03-18T22:50:54.259 INFO:teuthology.orchestra.run.smithi132.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 226 kB/s | 39 kB 00:00 2024-03-18T22:50:54.426 INFO:teuthology.orchestra.run.smithi132.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-18T22:50:54.451 INFO:teuthology.orchestra.run.smithi132.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-18T22:50:54.476 INFO:teuthology.orchestra.run.smithi132.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 260 kB/s | 56 kB 00:00 2024-03-18T22:50:54.501 INFO:teuthology.orchestra.run.smithi132.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-18T22:50:54.527 INFO:teuthology.orchestra.run.smithi132.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 835 kB/s | 63 kB 00:00 2024-03-18T22:50:54.548 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:54.549 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T22:50:54.552 INFO:teuthology.orchestra.run.smithi132.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-18T22:50:54.563 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T22:50:54.564 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T22:50:54.565 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T22:50:54.566 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:54.568 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T22:50:54.571 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T22:50:54.572 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T22:50:54.577 INFO:teuthology.orchestra.run.smithi132.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-18T22:50:54.603 INFO:teuthology.orchestra.run.smithi132.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-18T22:50:54.645 INFO:teuthology.orchestra.run.smithi132.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-18T22:50:54.652 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-18T22:50:54.669 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-18T22:50:54.670 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T22:50:54.671 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-18T22:50:54.672 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T22:50:54.673 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-18T22:50:54.674 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T22:50:54.675 INFO:teuthology.orchestra.run.smithi196.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T22:50:54.676 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T22:50:54.677 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T22:50:54.678 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T22:50:54.679 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T22:50:54.680 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T22:50:54.681 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T22:50:54.682 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T22:50:54.683 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================================ 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi196.stdout:Install 150 Packages 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 24 Packages 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:50:54.684 INFO:teuthology.orchestra.run.smithi132.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-18T22:50:54.685 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 224 M 2024-03-18T22:50:54.685 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-18T22:50:54.695 INFO:teuthology.orchestra.run.smithi132.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-18T22:50:54.870 INFO:teuthology.orchestra.run.smithi132.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 938 kB/s | 187 kB 00:00 2024-03-18T22:50:54.946 INFO:teuthology.orchestra.run.smithi132.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 780 kB/s | 195 kB 00:00 2024-03-18T22:50:54.971 INFO:teuthology.orchestra.run.smithi132.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-18T22:50:54.988 INFO:teuthology.orchestra.run.smithi132.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-03-18T22:50:55.013 INFO:teuthology.orchestra.run.smithi132.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 604 kB/s | 25 kB 00:00 2024-03-18T22:50:55.047 INFO:teuthology.orchestra.run.smithi132.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-03-18T22:50:55.139 INFO:teuthology.orchestra.run.smithi132.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-03-18T22:50:55.198 INFO:teuthology.orchestra.run.smithi132.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-03-18T22:50:55.239 INFO:teuthology.orchestra.run.smithi132.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 841 kB/s | 35 kB 00:00 2024-03-18T22:50:55.290 INFO:teuthology.orchestra.run.smithi132.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.4 MB/s | 216 kB 00:00 2024-03-18T22:50:55.307 INFO:teuthology.orchestra.run.smithi132.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2024-03-18T22:50:55.340 INFO:teuthology.orchestra.run.smithi132.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-18T22:50:55.560 INFO:teuthology.orchestra.run.smithi196.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-18T22:50:55.785 INFO:teuthology.orchestra.run.smithi196.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 3.9 MB/s | 891 kB 00:00 2024-03-18T22:50:55.860 INFO:teuthology.orchestra.run.smithi196.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-18T22:50:55.936 INFO:teuthology.orchestra.run.smithi196.stdout:(4/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 9.9 MB/s | 5.4 MB 00:00 2024-03-18T22:50:56.061 INFO:teuthology.orchestra.run.smithi196.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 12 MB/s | 1.5 MB 00:00 2024-03-18T22:50:56.136 INFO:teuthology.orchestra.run.smithi196.stdout:(6/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-03-18T22:50:56.387 INFO:teuthology.orchestra.run.smithi196.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 14 MB/s | 4.7 MB 00:00 2024-03-18T22:50:56.649 INFO:teuthology.orchestra.run.smithi132.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.5 MB/s | 4.1 MB 00:01 2024-03-18T22:50:57.112 INFO:teuthology.orchestra.run.smithi196.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 14 MB/s | 9.8 MB 00:00 2024-03-18T22:50:57.162 INFO:teuthology.orchestra.run.smithi196.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 506 kB/s | 25 kB 00:00 2024-03-18T22:50:57.174 INFO:teuthology.orchestra.run.smithi132.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-18T22:50:57.291 INFO:teuthology.orchestra.run.smithi132.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-03-18T22:50:57.383 INFO:teuthology.orchestra.run.smithi132.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-03-18T22:50:57.733 INFO:teuthology.orchestra.run.smithi132.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-03-18T22:50:57.833 INFO:teuthology.orchestra.run.smithi132.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-03-18T22:50:57.917 INFO:teuthology.orchestra.run.smithi132.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-03-18T22:50:58.009 INFO:teuthology.orchestra.run.smithi132.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-03-18T22:50:58.143 INFO:teuthology.orchestra.run.smithi132.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-03-18T22:50:58.190 INFO:teuthology.orchestra.run.smithi196.stdout:(10/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 8.2 MB/s | 17 MB 00:02 2024-03-18T22:50:58.274 INFO:teuthology.orchestra.run.smithi196.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.6 MB/s | 221 kB 00:00 2024-03-18T22:50:58.341 INFO:teuthology.orchestra.run.smithi196.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 407 kB/s | 27 kB 00:00 2024-03-18T22:50:58.368 INFO:teuthology.orchestra.run.smithi132.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-18T22:50:58.407 INFO:teuthology.orchestra.run.smithi196.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 477 kB/s | 32 kB 00:00 2024-03-18T22:50:58.566 INFO:teuthology.orchestra.run.smithi196.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 4.6 MB/s | 746 kB 00:00 2024-03-18T22:50:58.584 INFO:teuthology.orchestra.run.smithi132.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-03-18T22:50:58.650 INFO:teuthology.orchestra.run.smithi196.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.9 MB/s | 163 kB 00:00 2024-03-18T22:50:58.685 INFO:teuthology.orchestra.run.smithi132.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-03-18T22:50:58.741 INFO:teuthology.orchestra.run.smithi196.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.4 MB/s | 127 kB 00:00 2024-03-18T22:50:58.793 INFO:teuthology.orchestra.run.smithi132.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-03-18T22:50:58.894 INFO:teuthology.orchestra.run.smithi132.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-03-18T22:50:58.908 INFO:teuthology.orchestra.run.smithi196.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 2.9 MB/s | 501 kB 00:00 2024-03-18T22:50:58.994 INFO:teuthology.orchestra.run.smithi132.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-18T22:50:59.094 INFO:teuthology.orchestra.run.smithi132.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-18T22:50:59.142 INFO:teuthology.orchestra.run.smithi196.stdout:(18/174): ceph-common-19.0.0-2206.gb6ae2491.el9 5.5 MB/s | 21 MB 00:03 2024-03-18T22:50:59.203 INFO:teuthology.orchestra.run.smithi132.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-18T22:50:59.225 INFO:teuthology.orchestra.run.smithi196.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 542 kB/s | 45 kB 00:00 2024-03-18T22:50:59.334 INFO:teuthology.orchestra.run.smithi196.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.2 MB/s | 131 kB 00:00 2024-03-18T22:50:59.336 INFO:teuthology.orchestra.run.smithi132.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-03-18T22:50:59.434 INFO:teuthology.orchestra.run.smithi196.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.6 MB/s | 162 kB 00:00 2024-03-18T22:50:59.445 INFO:teuthology.orchestra.run.smithi132.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-18T22:50:59.543 INFO:teuthology.orchestra.run.smithi196.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 2.9 MB/s | 321 kB 00:00 2024-03-18T22:50:59.610 INFO:teuthology.orchestra.run.smithi196.stdout:(23/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 7.3 MB/s | 5.1 MB 00:00 2024-03-18T22:50:59.635 INFO:teuthology.orchestra.run.smithi196.stdout:(24/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.2 MB/s | 298 kB 00:00 2024-03-18T22:50:59.668 INFO:teuthology.orchestra.run.smithi196.stdout:(25/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.7 MB/s | 99 kB 00:00 2024-03-18T22:50:59.719 INFO:teuthology.orchestra.run.smithi196.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-18T22:50:59.819 INFO:teuthology.orchestra.run.smithi196.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-18T22:50:59.911 INFO:teuthology.orchestra.run.smithi196.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 370 kB/s | 34 kB 00:00 2024-03-18T22:51:00.020 INFO:teuthology.orchestra.run.smithi196.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.3 MB/s | 139 kB 00:00 2024-03-18T22:51:00.079 INFO:teuthology.orchestra.run.smithi132.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-03-18T22:51:00.187 INFO:teuthology.orchestra.run.smithi132.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-18T22:51:00.313 INFO:teuthology.orchestra.run.smithi132.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-03-18T22:51:00.413 INFO:teuthology.orchestra.run.smithi132.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-03-18T22:51:00.462 INFO:teuthology.orchestra.run.smithi196.stdout:(30/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 14 MB/s | 48 MB 00:03 2024-03-18T22:51:00.512 INFO:teuthology.orchestra.run.smithi196.stdout:(31/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 3.7 MB/s | 3.1 MB 00:00 2024-03-18T22:51:00.521 INFO:teuthology.orchestra.run.smithi132.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-03-18T22:51:00.562 INFO:teuthology.orchestra.run.smithi196.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 3.3 MB/s | 1.8 MB 00:00 2024-03-18T22:51:00.614 INFO:teuthology.orchestra.run.smithi132.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-03-18T22:51:00.637 INFO:teuthology.orchestra.run.smithi196.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2206.gb6 1.9 MB/s | 248 kB 00:00 2024-03-18T22:51:00.663 INFO:teuthology.orchestra.run.smithi196.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 493 kB/s | 49 kB 00:00 2024-03-18T22:51:00.697 INFO:teuthology.orchestra.run.smithi196.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2206.gb 281 kB/s | 16 kB 00:00 2024-03-18T22:51:00.705 INFO:teuthology.orchestra.run.smithi132.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-18T22:51:00.747 INFO:teuthology.orchestra.run.smithi196.stdout:(36/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 3.1 MB/s | 264 kB 00:00 2024-03-18T22:51:00.822 INFO:teuthology.orchestra.run.smithi132.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-03-18T22:51:00.839 INFO:teuthology.orchestra.run.smithi196.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 199 kB/s | 18 kB 00:00 2024-03-18T22:51:00.864 INFO:teuthology.orchestra.run.smithi196.stdout:(38/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 4.4 MB/s | 749 kB 00:00 2024-03-18T22:51:00.920 INFO:teuthology.orchestra.run.smithi132.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 477 kB/s | 46 kB 00:00 2024-03-18T22:51:00.948 INFO:teuthology.orchestra.run.smithi196.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-18T22:51:01.020 INFO:teuthology.orchestra.run.smithi132.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 789 kB/s | 79 kB 00:00 2024-03-18T22:51:01.098 INFO:teuthology.orchestra.run.smithi196.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-18T22:51:01.120 INFO:teuthology.orchestra.run.smithi132.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-03-18T22:51:01.132 INFO:teuthology.orchestra.run.smithi196.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-18T22:51:01.165 INFO:teuthology.orchestra.run.smithi196.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2024-03-18T22:51:01.190 INFO:teuthology.orchestra.run.smithi196.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-18T22:51:01.215 INFO:teuthology.orchestra.run.smithi196.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-18T22:51:01.221 INFO:teuthology.orchestra.run.smithi132.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-03-18T22:51:01.282 INFO:teuthology.orchestra.run.smithi196.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-18T22:51:01.299 INFO:teuthology.orchestra.run.smithi196.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 10 kB 00:00 2024-03-18T22:51:01.325 INFO:teuthology.orchestra.run.smithi196.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2024-03-18T22:51:01.350 INFO:teuthology.orchestra.run.smithi196.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-18T22:51:01.367 INFO:teuthology.orchestra.run.smithi196.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 657 kB/s | 44 kB 00:00 2024-03-18T22:51:01.380 INFO:teuthology.orchestra.run.smithi132.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 103 kB/s | 16 kB 00:00 2024-03-18T22:51:01.401 INFO:teuthology.orchestra.run.smithi196.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-18T22:51:01.417 INFO:teuthology.orchestra.run.smithi196.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2024-03-18T22:51:01.442 INFO:teuthology.orchestra.run.smithi196.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-18T22:51:01.476 INFO:teuthology.orchestra.run.smithi196.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-03-18T22:51:01.480 INFO:teuthology.orchestra.run.smithi132.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2024-03-18T22:51:01.518 INFO:teuthology.orchestra.run.smithi196.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-18T22:51:01.544 INFO:teuthology.orchestra.run.smithi196.stdout:(55/174): libtool-2.4.6-45.el9.x86_64.rpm 2.9 MB/s | 578 kB 00:00 2024-03-18T22:51:01.568 INFO:teuthology.orchestra.run.smithi196.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 372 kB/s | 34 kB 00:00 2024-03-18T22:51:01.588 INFO:teuthology.orchestra.run.smithi132.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-18T22:51:01.652 INFO:teuthology.orchestra.run.smithi196.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-18T22:51:01.680 INFO:teuthology.orchestra.run.smithi132.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 336 kB/s | 31 kB 00:00 2024-03-18T22:51:01.694 INFO:teuthology.orchestra.run.smithi196.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 6.5 MB/s | 811 kB 00:00 2024-03-18T22:51:01.736 INFO:teuthology.orchestra.run.smithi196.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-18T22:51:01.761 INFO:teuthology.orchestra.run.smithi196.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-18T22:51:01.780 INFO:teuthology.orchestra.run.smithi132.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-18T22:51:01.820 INFO:teuthology.orchestra.run.smithi196.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.4 MB/s | 78 kB 00:00 2024-03-18T22:51:01.853 INFO:teuthology.orchestra.run.smithi196.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-18T22:51:01.881 INFO:teuthology.orchestra.run.smithi132.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 592 kB/s | 59 kB 00:00 2024-03-18T22:51:01.954 INFO:teuthology.orchestra.run.smithi196.stdout:(63/174): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-03-18T22:51:01.973 INFO:teuthology.orchestra.run.smithi132.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-18T22:51:02.046 INFO:teuthology.orchestra.run.smithi196.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-18T22:51:02.065 INFO:teuthology.orchestra.run.smithi132.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-03-18T22:51:02.071 INFO:teuthology.orchestra.run.smithi196.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 114 kB/s | 24 kB 00:00 2024-03-18T22:51:02.096 INFO:teuthology.orchestra.run.smithi196.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 129 kB/s | 18 kB 00:00 2024-03-18T22:51:02.121 INFO:teuthology.orchestra.run.smithi196.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-18T22:51:02.147 INFO:teuthology.orchestra.run.smithi196.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-18T22:51:02.172 INFO:teuthology.orchestra.run.smithi196.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 217 kB 00:00 2024-03-18T22:51:02.173 INFO:teuthology.orchestra.run.smithi132.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-18T22:51:02.272 INFO:teuthology.orchestra.run.smithi196.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-03-18T22:51:02.273 INFO:teuthology.orchestra.run.smithi132.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 898 kB/s | 90 kB 00:00 2024-03-18T22:51:02.298 INFO:teuthology.orchestra.run.smithi196.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-18T22:51:02.323 INFO:teuthology.orchestra.run.smithi196.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 463 kB/s | 69 kB 00:00 2024-03-18T22:51:02.348 INFO:teuthology.orchestra.run.smithi196.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 548 kB/s | 41 kB 00:00 2024-03-18T22:51:02.373 INFO:teuthology.orchestra.run.smithi196.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-18T22:51:02.374 INFO:teuthology.orchestra.run.smithi132.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-18T22:51:02.399 INFO:teuthology.orchestra.run.smithi196.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-18T22:51:02.432 INFO:teuthology.orchestra.run.smithi196.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-18T22:51:02.466 INFO:teuthology.orchestra.run.smithi132.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-03-18T22:51:02.516 INFO:teuthology.orchestra.run.smithi196.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.2 MB/s | 211 kB 00:00 2024-03-18T22:51:02.541 INFO:teuthology.orchestra.run.smithi196.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 283 kB/s | 47 kB 00:00 2024-03-18T22:51:02.575 INFO:teuthology.orchestra.run.smithi132.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-18T22:51:02.792 INFO:teuthology.orchestra.run.smithi196.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-18T22:51:02.816 INFO:teuthology.orchestra.run.smithi196.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 129 kB/s | 39 kB 00:00 2024-03-18T22:51:02.842 INFO:teuthology.orchestra.run.smithi196.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 137 kB/s | 41 kB 00:00 2024-03-18T22:51:02.867 INFO:teuthology.orchestra.run.smithi196.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-18T22:51:02.892 INFO:teuthology.orchestra.run.smithi196.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T22:51:02.917 INFO:teuthology.orchestra.run.smithi196.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 839 kB/s | 63 kB 00:00 2024-03-18T22:51:02.942 INFO:teuthology.orchestra.run.smithi196.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-03-18T22:51:02.968 INFO:teuthology.orchestra.run.smithi196.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-18T22:51:02.993 INFO:teuthology.orchestra.run.smithi196.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-18T22:51:03.027 INFO:teuthology.orchestra.run.smithi196.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-18T22:51:03.033 INFO:teuthology.orchestra.run.smithi132.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-03-18T22:51:03.052 INFO:teuthology.orchestra.run.smithi196.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-18T22:51:03.086 INFO:teuthology.orchestra.run.smithi196.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-18T22:51:03.111 INFO:teuthology.orchestra.run.smithi196.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-18T22:51:03.303 INFO:teuthology.orchestra.run.smithi196.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 749 kB/s | 187 kB 00:00 2024-03-18T22:51:03.328 INFO:teuthology.orchestra.run.smithi196.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 900 kB/s | 195 kB 00:00 2024-03-18T22:51:03.350 INFO:teuthology.orchestra.run.smithi132.stdout:(149/174): librados2-19.0.0-2206.gb6ae2491.el9. 11 MB/s | 3.4 MB 00:00 2024-03-18T22:51:03.403 INFO:teuthology.orchestra.run.smithi196.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-03-18T22:51:03.429 INFO:teuthology.orchestra.run.smithi196.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 994 kB/s | 125 kB 00:00 2024-03-18T22:51:03.446 INFO:teuthology.orchestra.run.smithi196.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 605 kB/s | 25 kB 00:00 2024-03-18T22:51:03.529 INFO:teuthology.orchestra.run.smithi196.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-03-18T22:51:03.542 INFO:teuthology.orchestra.run.smithi132.stdout:(150/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 16 MB/s | 3.1 MB 00:00 2024-03-18T22:51:03.571 INFO:teuthology.orchestra.run.smithi196.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-18T22:51:03.584 INFO:teuthology.orchestra.run.smithi132.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-18T22:51:03.596 INFO:teuthology.orchestra.run.smithi196.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-03-18T22:51:03.638 INFO:teuthology.orchestra.run.smithi196.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 839 kB/s | 35 kB 00:00 2024-03-18T22:51:03.705 INFO:teuthology.orchestra.run.smithi196.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 15 MB/s | 4.1 MB 00:00 2024-03-18T22:51:03.722 INFO:teuthology.orchestra.run.smithi196.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 532 kB/s | 44 kB 00:00 2024-03-18T22:51:03.748 INFO:teuthology.orchestra.run.smithi196.stdout:(103/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-18T22:51:03.765 INFO:teuthology.orchestra.run.smithi196.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 531 kB/s | 22 kB 00:00 2024-03-18T22:51:03.790 INFO:teuthology.orchestra.run.smithi196.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-18T22:51:03.865 INFO:teuthology.orchestra.run.smithi196.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-03-18T22:51:03.957 INFO:teuthology.orchestra.run.smithi196.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-03-18T22:51:03.991 INFO:teuthology.orchestra.run.smithi196.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-03-18T22:51:04.016 INFO:teuthology.orchestra.run.smithi196.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-03-18T22:51:04.034 INFO:teuthology.orchestra.run.smithi132.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.1 MB/s | 2.3 MB 00:00 2024-03-18T22:51:04.041 INFO:teuthology.orchestra.run.smithi196.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-03-18T22:51:04.075 INFO:teuthology.orchestra.run.smithi196.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-03-18T22:51:04.100 INFO:teuthology.orchestra.run.smithi196.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-18T22:51:04.125 INFO:teuthology.orchestra.run.smithi196.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-03-18T22:51:04.217 INFO:teuthology.orchestra.run.smithi196.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-03-18T22:51:04.243 INFO:teuthology.orchestra.run.smithi196.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-03-18T22:51:04.276 INFO:teuthology.orchestra.run.smithi196.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-03-18T22:51:04.285 INFO:teuthology.orchestra.run.smithi132.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 7.3 MB/s | 1.8 MB 00:00 2024-03-18T22:51:04.302 INFO:teuthology.orchestra.run.smithi196.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-03-18T22:51:04.327 INFO:teuthology.orchestra.run.smithi196.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-03-18T22:51:04.335 INFO:teuthology.orchestra.run.smithi132.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 842 kB/s | 42 kB 00:00 2024-03-18T22:51:04.352 INFO:teuthology.orchestra.run.smithi196.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-03-18T22:51:04.394 INFO:teuthology.orchestra.run.smithi132.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-18T22:51:04.394 INFO:teuthology.orchestra.run.smithi196.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-18T22:51:04.436 INFO:teuthology.orchestra.run.smithi196.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-18T22:51:04.444 INFO:teuthology.orchestra.run.smithi132.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-18T22:51:04.461 INFO:teuthology.orchestra.run.smithi196.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-03-18T22:51:04.486 INFO:teuthology.orchestra.run.smithi196.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-03-18T22:51:04.494 INFO:teuthology.orchestra.run.smithi132.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 731 kB/s | 36 kB 00:00 2024-03-18T22:51:04.528 INFO:teuthology.orchestra.run.smithi196.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-03-18T22:51:04.561 INFO:teuthology.orchestra.run.smithi132.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-18T22:51:04.612 INFO:teuthology.orchestra.run.smithi196.stdout:(125/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-03-18T22:51:04.619 INFO:teuthology.orchestra.run.smithi132.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-18T22:51:04.637 INFO:teuthology.orchestra.run.smithi196.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-03-18T22:51:04.663 INFO:teuthology.orchestra.run.smithi196.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-18T22:51:04.670 INFO:teuthology.orchestra.run.smithi132.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 460 kB/s | 23 kB 00:00 2024-03-18T22:51:04.688 INFO:teuthology.orchestra.run.smithi196.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-18T22:51:04.713 INFO:teuthology.orchestra.run.smithi196.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-18T22:51:04.720 INFO:teuthology.orchestra.run.smithi132.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-18T22:51:04.739 INFO:teuthology.orchestra.run.smithi196.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-18T22:51:04.779 INFO:teuthology.orchestra.run.smithi132.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-18T22:51:04.789 INFO:teuthology.orchestra.run.smithi196.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-03-18T22:51:04.814 INFO:teuthology.orchestra.run.smithi196.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-03-18T22:51:04.837 INFO:teuthology.orchestra.run.smithi132.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-18T22:51:04.840 INFO:teuthology.orchestra.run.smithi196.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-18T22:51:04.865 INFO:teuthology.orchestra.run.smithi196.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-18T22:51:04.887 INFO:teuthology.orchestra.run.smithi132.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-18T22:51:04.890 INFO:teuthology.orchestra.run.smithi196.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-18T22:51:04.915 INFO:teuthology.orchestra.run.smithi196.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-18T22:51:04.938 INFO:teuthology.orchestra.run.smithi132.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 575 kB/s | 29 kB 00:00 2024-03-18T22:51:04.949 INFO:teuthology.orchestra.run.smithi196.stdout:(137/174): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-03-18T22:51:04.974 INFO:teuthology.orchestra.run.smithi196.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-18T22:51:04.991 INFO:teuthology.orchestra.run.smithi196.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-03-18T22:51:05.008 INFO:teuthology.orchestra.run.smithi196.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-03-18T22:51:05.033 INFO:teuthology.orchestra.run.smithi196.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-03-18T22:51:05.051 INFO:teuthology.orchestra.run.smithi196.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-03-18T22:51:05.067 INFO:teuthology.orchestra.run.smithi196.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-03-18T22:51:05.071 INFO:teuthology.orchestra.run.smithi132.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 9.1 MB/s | 1.2 MB 00:00 2024-03-18T22:51:05.084 INFO:teuthology.orchestra.run.smithi196.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-18T22:51:05.109 INFO:teuthology.orchestra.run.smithi196.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-03-18T22:51:05.135 INFO:teuthology.orchestra.run.smithi196.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-18T22:51:05.152 INFO:teuthology.orchestra.run.smithi196.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-03-18T22:51:05.177 INFO:teuthology.orchestra.run.smithi196.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-18T22:51:05.219 INFO:teuthology.orchestra.run.smithi196.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-18T22:51:05.263 INFO:teuthology.orchestra.run.smithi132.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-18T22:51:05.369 INFO:teuthology.orchestra.run.smithi196.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-03-18T22:51:05.411 INFO:teuthology.orchestra.run.smithi196.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-18T22:51:05.455 INFO:teuthology.orchestra.run.smithi132.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-03-18T22:51:05.514 INFO:teuthology.orchestra.run.smithi132.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-03-18T22:51:05.564 INFO:teuthology.orchestra.run.smithi132.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 623 kB/s | 31 kB 00:00 2024-03-18T22:51:05.673 INFO:teuthology.orchestra.run.smithi132.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-18T22:51:05.723 INFO:teuthology.orchestra.run.smithi132.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.5 MB/s | 74 kB 00:00 2024-03-18T22:51:05.745 INFO:teuthology.orchestra.run.smithi196.stdout:(152/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-03-18T22:51:05.795 INFO:teuthology.orchestra.run.smithi196.stdout:(153/174): librados2-19.0.0-2206.gb6ae2491.el9. 5.5 MB/s | 3.4 MB 00:00 2024-03-18T22:51:05.845 INFO:teuthology.orchestra.run.smithi196.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 846 kB/s | 42 kB 00:00 2024-03-18T22:51:05.979 INFO:teuthology.orchestra.run.smithi196.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-03-18T22:51:06.037 INFO:teuthology.orchestra.run.smithi196.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-18T22:51:06.079 INFO:teuthology.orchestra.run.smithi196.stdout:(157/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.5 MB/s | 2.3 MB 00:00 2024-03-18T22:51:06.104 INFO:teuthology.orchestra.run.smithi196.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 544 kB/s | 36 kB 00:00 2024-03-18T22:51:06.163 INFO:teuthology.orchestra.run.smithi196.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-18T22:51:06.213 INFO:teuthology.orchestra.run.smithi196.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 460 kB/s | 23 kB 00:00 2024-03-18T22:51:06.247 INFO:teuthology.orchestra.run.smithi196.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-18T22:51:06.272 INFO:teuthology.orchestra.run.smithi196.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-18T22:51:06.314 INFO:teuthology.orchestra.run.smithi196.stdout:(163/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.2 MB/s | 1.8 MB 00:00 2024-03-18T22:51:06.340 INFO:teuthology.orchestra.run.smithi196.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-18T22:51:06.365 INFO:teuthology.orchestra.run.smithi196.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-03-18T22:51:06.390 INFO:teuthology.orchestra.run.smithi196.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 846 kB/s | 64 kB 00:00 2024-03-18T22:51:06.416 INFO:teuthology.orchestra.run.smithi196.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-18T22:51:06.749 INFO:teuthology.orchestra.run.smithi196.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 2.2 MB 00:00 2024-03-18T22:51:06.791 INFO:teuthology.orchestra.run.smithi196.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.1 MB/s | 2.3 MB 00:00 2024-03-18T22:51:06.825 INFO:teuthology.orchestra.run.smithi196.stdout:(170/174): openssl-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 1.2 MB 00:00 2024-03-18T22:51:06.850 INFO:teuthology.orchestra.run.smithi196.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-03-18T22:51:06.875 INFO:teuthology.orchestra.run.smithi196.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-18T22:51:06.934 INFO:teuthology.orchestra.run.smithi196.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 892 kB/s | 74 kB 00:00 2024-03-18T22:51:07.167 INFO:teuthology.orchestra.run.smithi196.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.1 MB/s | 1.1 MB 00:00 2024-03-18T22:51:07.171 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:51:07.171 INFO:teuthology.orchestra.run.smithi196.stdout:Total 18 MB/s | 224 MB 00:12 2024-03-18T22:51:07.658 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-18T22:51:07.927 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-18T22:51:07.927 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-18T22:51:09.643 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-18T22:51:09.644 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-18T22:51:16.513 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-18T22:51:16.612 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:51:16.646 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:51:16.752 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:51:16.762 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:51:16.902 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:51:16.955 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:51:16.989 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:51:17.023 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:51:17.074 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:51:17.412 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:51:17.466 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:51:17.572 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:51:17.609 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:51:17.684 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:51:17.713 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:51:17.749 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:51:17.915 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:51:17.961 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:51:17.989 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:51:18.118 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:51:18.140 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:51:18.204 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:51:18.252 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:51:18.275 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:51:18.292 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:51:18.433 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:51:18.464 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:51:18.504 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:51:18.610 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:51:18.724 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:51:18.767 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:51:18.828 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:51:18.885 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:51:19.091 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:51:19.199 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:51:19.293 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:51:19.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:51:19.358 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:51:19.394 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:51:19.468 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:51:19.522 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:51:19.560 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:51:19.627 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:51:19.659 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:51:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:51:19.735 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:51:19.763 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:51:19.833 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:51:19.868 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:51:19.911 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:51:19.940 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:51:20.009 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:51:20.049 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:51:20.122 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:51:20.257 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:51:20.310 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:51:20.830 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:51:20.914 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:51:21.000 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:51:21.058 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:51:21.067 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:51:21.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:51:21.139 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:51:21.177 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:51:21.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:51:21.429 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:51:21.585 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:51:21.667 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:51:21.711 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:51:21.847 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:51:21.889 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:51:21.944 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:51:22.272 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:51:22.316 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:51:22.353 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:51:22.400 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:51:22.435 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:51:22.498 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:51:22.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:51:22.683 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:51:22.741 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:51:22.764 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:51:22.805 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:51:22.899 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:51:22.939 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:51:22.979 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:51:23.004 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:51:23.029 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:51:23.103 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:51:23.230 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:51:23.332 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:51:23.379 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:51:23.427 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:51:23.483 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:51:23.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:51:23.717 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:51:23.846 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:51:23.890 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:51:23.922 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:51:23.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:51:23.980 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:51:24.015 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:51:24.054 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:51:24.086 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:51:24.115 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:51:24.191 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:51:24.226 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:51:24.259 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:51:24.300 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:51:24.327 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:51:24.415 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:51:24.497 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:51:24.541 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:51:24.573 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:51:24.619 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:51:24.717 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:51:24.766 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:51:25.488 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:51:25.538 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:51:25.572 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:51:25.620 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:51:25.709 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:51:25.740 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:51:25.917 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:51:25.946 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:51:26.023 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:51:26.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:51:26.778 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:51:28.003 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:51:28.080 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:51:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:51:28.142 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:51:28.169 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:51:28.483 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:51:28.516 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:51:28.590 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:51:28.621 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:51:28.657 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:51:29.118 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:51:29.145 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:51:29.215 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:51:29.238 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:51:31.433 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:51:31.468 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:51:32.009 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:51:32.046 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:51:32.110 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:51:32.111 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:51:32.111 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:51:32.111 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:32.156 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-03-18T22:51:43.983 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-03-18T22:51:43.984 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-03-18T22:51:43.984 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:44.146 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:51:44.162 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:51:44.162 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:44.162 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:51:44.163 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:45.632 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:51:45.658 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:51:45.659 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:45.659 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:51:45.659 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:51:45.659 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:51:45.659 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:45.883 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:51:45.906 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:51:45.906 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:45.906 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:51:45.906 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:51:45.907 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:51:45.907 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:46.343 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:51:46.366 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:46.452 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:51:46.828 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:51:46.859 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:51:46.891 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:51:46.943 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:51:46.965 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:51:47.200 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:51:47.233 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:51:48.197 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:51:48.225 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:51:48.326 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:51:48.480 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:51:48.503 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:51:48.526 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:48.570 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:51:48.581 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:51:48.626 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:51:49.499 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:51:49.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:51:49.553 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:51:49.553 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:49.553 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:51:49.553 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:51:49.553 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:51:49.554 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:49.597 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:51:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:51:49.620 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:49.620 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:51:49.620 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:49.665 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:51:49.688 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:49.990 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:51:50.013 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:51:50.013 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:51:50.013 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:51:50.014 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:51:50.014 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:51:50.014 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:51:54.427 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:51:54.469 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:51:54.499 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:51:54.526 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:51:54.708 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:51:54.818 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:51:54.860 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:51:54.898 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:51:54.942 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:51:54.985 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:51:55.010 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:51:55.042 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:51:55.080 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:51:55.126 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:51:55.187 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:51:55.212 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:51:55.785 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:51:55.826 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:51:55.846 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:51:56.666 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:51:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:51:56.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:51:56.867 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:51:56.900 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:51:56.925 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:51:56.951 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:51:56.977 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:51:56.978 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:51:57.065 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:51:57.095 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:51:57.095 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:51:57.171 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:51:57.197 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:51:57.230 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:51:57.256 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:51:57.289 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:51:57.322 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:51:57.370 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:51:57.411 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:51:57.436 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:51:57.461 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:51:57.486 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:51:57.511 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:51:57.519 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:51:57.519 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:51:57.706 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:51:57.706 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:51:57.798 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:51:57.822 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:51:57.847 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:51:57.880 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:51:57.907 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:51:57.970 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:52:02.926 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:52:02.927 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:52:02.928 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:52:02.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:52:02.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:52:02.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:52:02.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:52:02.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:52:02.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:52:02.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:52:02.948 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:52:02.949 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:52:02.950 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:52:02.951 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:52:02.952 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:52:02.953 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:52:02.954 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:52:02.955 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:52:02.956 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:52:02.957 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:52:02.957 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:52:02.957 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:52:06.005 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:52:06.006 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.007 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.008 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.009 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:52:06.010 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:52:06.011 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:52:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:52:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:52:06.014 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:52:06.015 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:52:06.016 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:52:06.017 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:52:06.018 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T22:52:06.019 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-18T22:52:06.464 DEBUG:teuthology.parallel:result is None 2024-03-18T22:53:03.050 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] jq-1.6-16.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/jq-1.6-16.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:53:03.051 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:53:03.051 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:53:03.548 INFO:teuthology.orchestra.run.smithi144.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.4 kB/s | 187 kB 02:12 2024-03-18T22:53:03.631 INFO:teuthology.orchestra.run.smithi144.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 kB/s | 195 kB 02:12 2024-03-18T22:53:03.715 INFO:teuthology.orchestra.run.smithi144.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 258 kB/s | 21 kB 00:00 2024-03-18T22:53:03.741 INFO:teuthology.orchestra.run.smithi144.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 648 kB/s | 125 kB 00:00 2024-03-18T22:53:03.766 INFO:teuthology.orchestra.run.smithi144.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-03-18T22:53:03.850 INFO:teuthology.orchestra.run.smithi144.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-03-18T22:53:03.900 INFO:teuthology.orchestra.run.smithi144.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-03-18T22:53:03.942 INFO:teuthology.orchestra.run.smithi144.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 216 kB 00:00 2024-03-18T22:53:04.017 INFO:teuthology.orchestra.run.smithi144.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 15 MB/s | 4.1 MB 00:00 2024-03-18T22:53:04.042 INFO:teuthology.orchestra.run.smithi144.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 348 kB/s | 35 kB 00:00 2024-03-18T22:53:04.067 INFO:teuthology.orchestra.run.smithi144.stdout:(102/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.0 kB/s | 803 kB 02:13 2024-03-18T22:53:04.092 INFO:teuthology.orchestra.run.smithi144.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 592 kB/s | 44 kB 00:00 2024-03-18T22:53:04.118 INFO:teuthology.orchestra.run.smithi144.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-03-18T22:53:04.201 INFO:teuthology.orchestra.run.smithi144.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-03-18T22:53:04.402 INFO:teuthology.orchestra.run.smithi144.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-03-18T22:53:04.435 INFO:teuthology.orchestra.run.smithi144.stdout:(107/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-18T22:53:04.461 INFO:teuthology.orchestra.run.smithi144.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-18T22:53:04.486 INFO:teuthology.orchestra.run.smithi144.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-03-18T22:53:04.519 INFO:teuthology.orchestra.run.smithi144.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-03-18T22:53:04.561 INFO:teuthology.orchestra.run.smithi144.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-03-18T22:53:04.587 INFO:teuthology.orchestra.run.smithi144.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-18T22:53:04.637 INFO:teuthology.orchestra.run.smithi144.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-18T22:53:04.721 INFO:teuthology.orchestra.run.smithi144.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-18T22:53:04.746 INFO:teuthology.orchestra.run.smithi144.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-03-18T22:53:04.779 INFO:teuthology.orchestra.run.smithi144.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-03-18T22:53:04.805 INFO:teuthology.orchestra.run.smithi144.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-18T22:53:04.839 INFO:teuthology.orchestra.run.smithi144.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-18T22:53:04.863 INFO:teuthology.orchestra.run.smithi144.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-18T22:53:04.939 INFO:teuthology.orchestra.run.smithi144.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-03-18T22:53:04.964 INFO:teuthology.orchestra.run.smithi144.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-18T22:53:04.990 INFO:teuthology.orchestra.run.smithi144.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-18T22:53:05.015 INFO:teuthology.orchestra.run.smithi144.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-03-18T22:53:05.040 INFO:teuthology.orchestra.run.smithi144.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-18T22:53:05.082 INFO:teuthology.orchestra.run.smithi144.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-18T22:53:05.108 INFO:teuthology.orchestra.run.smithi144.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-18T22:53:05.166 INFO:teuthology.orchestra.run.smithi144.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-18T22:53:05.183 INFO:teuthology.orchestra.run.smithi144.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-18T22:53:05.200 INFO:teuthology.orchestra.run.smithi144.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-03-18T22:53:05.217 INFO:teuthology.orchestra.run.smithi144.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-03-18T22:53:05.251 INFO:teuthology.orchestra.run.smithi144.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-03-18T22:53:05.293 INFO:teuthology.orchestra.run.smithi144.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-18T22:53:05.309 INFO:teuthology.orchestra.run.smithi144.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 501 kB/s | 46 kB 00:00 2024-03-18T22:53:05.334 INFO:teuthology.orchestra.run.smithi144.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-03-18T22:53:05.351 INFO:teuthology.orchestra.run.smithi144.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-03-18T22:53:05.377 INFO:teuthology.orchestra.run.smithi144.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-03-18T22:53:05.402 INFO:teuthology.orchestra.run.smithi144.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-03-18T22:53:05.427 INFO:teuthology.orchestra.run.smithi144.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-18T22:53:05.452 INFO:teuthology.orchestra.run.smithi144.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-18T22:53:05.478 INFO:teuthology.orchestra.run.smithi144.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-03-18T22:53:05.511 INFO:teuthology.orchestra.run.smithi144.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-18T22:53:05.537 INFO:teuthology.orchestra.run.smithi144.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-03-18T22:53:05.562 INFO:teuthology.orchestra.run.smithi144.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-03-18T22:53:05.587 INFO:teuthology.orchestra.run.smithi144.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-03-18T22:53:05.612 INFO:teuthology.orchestra.run.smithi144.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-18T22:53:05.638 INFO:teuthology.orchestra.run.smithi144.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-18T22:53:05.663 INFO:teuthology.orchestra.run.smithi144.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-18T22:53:05.688 INFO:teuthology.orchestra.run.smithi144.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-18T22:53:05.713 INFO:teuthology.orchestra.run.smithi144.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-18T22:53:05.755 INFO:teuthology.orchestra.run.smithi144.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-18T22:53:05.956 INFO:teuthology.orchestra.run.smithi144.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2024-03-18T22:53:06.041 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-1.47.2-1.el9.x86_64.rpm: Status code: 404 for http://mirror.cpsc.ucalgary.ca/mirror/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-1.47.2-1.el9.x86_64.rpm (IP: 136.159.208.97) 2024-03-18T22:53:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:(152/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-03-18T22:53:06.312 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:53:06.312 INFO:teuthology.orchestra.run.smithi132.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:53:06.615 INFO:teuthology.orchestra.run.smithi144.stdout:(153/174): librados2-19.0.0-2206.gb6ae2491.el9. 3.7 MB/s | 3.4 MB 00:00 2024-03-18T22:53:06.673 INFO:teuthology.orchestra.run.smithi132.stdout:(173/174): lua-devel-5.4.4-4.el9.x86_64.rpm 174 B/s | 22 kB 02:11 2024-03-18T22:53:07.448 INFO:teuthology.orchestra.run.smithi132.stdout:(174/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.5 kB/s | 863 kB 02:12 2024-03-18T22:53:07.449 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:53:07.450 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.6 MB/s | 224 MB 02:24 2024-03-18T22:53:07.900 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-03-18T22:53:08.177 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-03-18T22:53:08.178 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-03-18T22:53:10.278 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-03-18T22:53:10.278 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-03-18T22:53:17.224 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-03-18T22:53:17.321 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:53:17.364 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:53:17.402 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:53:17.411 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:53:17.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:53:17.522 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:53:17.555 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:53:17.589 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:53:17.641 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:53:17.972 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:53:18.034 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:53:18.156 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:53:18.185 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:53:18.268 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:53:18.302 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:53:18.345 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:53:18.535 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:53:18.604 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:53:18.633 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:53:18.736 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:53:18.759 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:53:18.815 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:53:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:53:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:53:18.900 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:53:19.035 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:53:19.067 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:53:19.106 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:53:19.296 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:53:19.452 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:53:19.511 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:53:19.609 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:53:19.662 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:53:19.864 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:53:19.983 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:53:20.078 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:53:20.109 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:53:20.136 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:53:20.188 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:53:20.270 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:53:20.324 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:53:20.362 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:53:20.426 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:53:20.462 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:53:20.505 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:53:20.538 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:53:20.566 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:53:20.635 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:53:20.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:53:20.723 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:53:20.751 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:53:20.820 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:53:20.866 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:53:20.939 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:53:21.072 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:53:21.118 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:53:21.644 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:53:21.721 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:53:21.807 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:53:21.842 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:53:21.850 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:53:21.884 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:53:21.931 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:53:21.969 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:53:22.000 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:53:22.278 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:53:22.469 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:53:22.587 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:53:22.756 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:53:23.015 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:53:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:53:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:53:23.301 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:53:23.358 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:53:23.395 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:53:23.441 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:53:23.477 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:53:23.520 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:53:23.571 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:53:23.703 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:53:23.751 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:53:23.782 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:53:23.823 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:53:23.918 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:53:23.957 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:53:24.002 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:53:24.039 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:53:24.064 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:53:24.168 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:53:24.298 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:53:24.405 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:53:24.448 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:53:24.496 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:53:24.560 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:53:24.770 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:53:24.821 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:53:24.950 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:53:24.987 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:53:25.018 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:53:25.058 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:53:25.086 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:53:25.128 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:53:25.176 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:53:25.207 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:53:25.238 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:53:25.313 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:53:25.348 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:53:25.381 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:53:25.423 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:53:25.458 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:53:25.545 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:53:25.636 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:53:25.689 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:53:25.720 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:53:25.767 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:53:25.863 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:53:25.913 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:53:26.638 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:53:26.686 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:53:26.720 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:53:26.767 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:53:26.846 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:53:26.895 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:53:27.071 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:53:27.102 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:53:27.179 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:53:27.816 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:53:27.940 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:53:29.185 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:53:29.277 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:53:29.309 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:53:29.340 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:53:29.367 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:53:29.680 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:53:29.706 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:53:29.796 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:53:29.827 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:53:29.864 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:53:30.322 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:53:30.351 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:53:30.404 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:53:30.436 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:53:32.702 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:53:32.741 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:53:33.284 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:53:33.328 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:53:33.411 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:53:33.411 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:53:33.411 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:53:33.411 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:33.446 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:53:45.197 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:53:45.197 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-03-18T22:53:45.197 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-03-18T22:53:45.197 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-03-18T22:53:45.198 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-03-18T22:53:45.198 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-03-18T22:53:45.198 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-03-18T22:53:45.198 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-03-18T22:53:45.198 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:45.361 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:53:45.375 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:53:45.375 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:45.375 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:53:45.376 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:46.863 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:53:46.888 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:53:46.889 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:46.889 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:53:46.889 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:53:46.889 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:53:46.889 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:47.114 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:53:47.136 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:47.573 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:53:47.595 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:47.691 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:53:48.142 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:53:48.173 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:53:48.205 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:53:48.265 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:53:48.288 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:53:48.521 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:53:48.556 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:53:49.518 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:53:49.548 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:53:49.649 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:53:49.810 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:53:49.850 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:53:49.873 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:53:49.873 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:49.873 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:53:49.873 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:53:49.873 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:53:49.874 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:49.917 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:53:49.928 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:53:49.983 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:53:50.851 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:53:52.432 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:53:52.432 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:53:52.432 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:52.432 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:53:52.432 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:53:52.433 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:53:52.434 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:53:52.435 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:53:52.435 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:53:52.435 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:53:52.435 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:53:55.828 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:53:55.869 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:53:55.900 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:53:55.927 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:53:56.118 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:53:56.236 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:53:56.275 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:53:56.328 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:53:56.384 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:53:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:53:56.460 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:53:56.500 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:53:56.530 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:53:56.601 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:53:56.674 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:53:56.711 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:53:57.278 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:53:57.318 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:53:57.337 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:53:58.154 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:53:58.247 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:53:58.289 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:53:58.384 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:53:58.417 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:53:58.442 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:53:58.468 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:53:58.494 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:53:58.494 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:53:58.582 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:53:58.620 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:53:58.620 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:53:58.705 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:53:58.732 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:53:58.764 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:53:58.790 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:53:58.815 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:53:58.849 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:53:58.888 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:53:58.919 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:53:58.945 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:53:58.970 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:53:58.995 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:53:59.012 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:53:59.018 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:53:59.018 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:53:59.199 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:53:59.199 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:53:59.265 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:53:59.290 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:53:59.315 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:53:59.348 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:53:59.374 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:53:59.441 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:54:04.443 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:54:04.444 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:54:04.445 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:54:04.446 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:54:04.446 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:54:04.446 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:54:04.462 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:54:04.462 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:54:04.462 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:54:04.462 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:54:04.463 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:54:04.464 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:54:04.465 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:54:04.466 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:54:04.467 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:54:04.468 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:54:04.469 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:54:04.470 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:54:04.471 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:54:04.472 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:54:04.473 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:54:04.474 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:54:04.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:54:04.476 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:54:04.477 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:54:04.478 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:54:04.479 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:54:04.480 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:54:07.475 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:54:07.475 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.476 INFO:teuthology.orchestra.run.smithi132.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:54:07.477 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.478 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.479 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:54:07.480 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:54:07.481 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:54:07.482 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:54:07.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:54:07.484 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:54:07.485 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.486 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:54:07.487 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:54:07.488 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:54:07.489 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:54:07.489 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:54:07.489 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-03-18T22:54:08.030 DEBUG:teuthology.parallel:result is None 2024-03-18T22:55:18.416 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-1.47.2-1.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-1.47.2-1.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-18T22:55:18.416 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-libnm-1.47.2-1.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-libnm-1.47.2-1.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-18T22:55:18.417 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-team-1.47.2-1.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.47.2-1.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-18T22:55:18.598 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-libnm-1.47.2-1.el9.x86_64.rpm: Status code: 404 for http://mirror.cpsc.ucalgary.ca/mirror/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-libnm-1.47.2-1.el9.x86_64.rpm (IP: 136.159.208.97) 2024-03-18T22:55:18.599 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-team-1.47.2-1.el9.x86_64.rpm: Status code: 404 for http://mirror.cpsc.ucalgary.ca/mirror/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.47.2-1.el9.x86_64.rpm (IP: 136.159.208.97) 2024-03-18T22:55:18.735 INFO:teuthology.orchestra.run.smithi144.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 326 B/s | 42 kB 02:12 2024-03-18T22:55:18.868 INFO:teuthology.orchestra.run.smithi144.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.8 MB/s | 248 kB 00:00 2024-03-18T22:55:19.043 INFO:teuthology.orchestra.run.smithi144.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 407 kB/s | 71 kB 00:00 2024-03-18T22:55:19.094 INFO:teuthology.orchestra.run.smithi144.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 730 kB/s | 36 kB 00:00 2024-03-18T22:55:19.186 INFO:teuthology.orchestra.run.smithi144.stdout:(158/174): NetworkManager-libnm-1.47.2-1.el9.x8 14 kB/s | 1.8 MB 02:12 2024-03-18T22:55:19.211 INFO:teuthology.orchestra.run.smithi144.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-03-18T22:55:19.278 INFO:teuthology.orchestra.run.smithi144.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-03-18T22:55:19.303 INFO:teuthology.orchestra.run.smithi144.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-18T22:55:19.328 INFO:teuthology.orchestra.run.smithi144.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-18T22:55:19.387 INFO:teuthology.orchestra.run.smithi144.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-18T22:55:19.412 INFO:teuthology.orchestra.run.smithi144.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-18T22:55:19.437 INFO:teuthology.orchestra.run.smithi144.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-18T22:55:19.463 INFO:teuthology.orchestra.run.smithi144.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-18T22:55:20.155 INFO:teuthology.orchestra.run.smithi144.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 1.7 MB/s | 1.2 MB 00:00 2024-03-18T22:55:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.8 MB/s | 2.2 MB 00:00 2024-03-18T22:55:20.363 INFO:teuthology.orchestra.run.smithi144.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-18T22:55:20.405 INFO:teuthology.orchestra.run.smithi144.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 750 kB/s | 31 kB 00:00 2024-03-18T22:55:20.672 INFO:teuthology.orchestra.run.smithi144.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-03-18T22:55:20.773 INFO:teuthology.orchestra.run.smithi144.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 744 kB/s | 74 kB 00:00 2024-03-18T22:55:21.073 INFO:teuthology.orchestra.run.smithi144.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 2.3 MB 00:00 2024-03-18T22:57:29.655 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] NetworkManager-1.47.2-1.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-1.47.2-1.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-18T22:57:30.368 INFO:teuthology.orchestra.run.smithi144.stdout:(174/174): NetworkManager-1.47.2-1.el9.x86_64.r 8.9 kB/s | 2.3 MB 04:24 2024-03-18T22:57:30.369 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:57:30.369 INFO:teuthology.orchestra.run.smithi144.stdout:Total 562 kB/s | 224 MB 06:48 2024-03-18T22:57:30.821 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-18T22:57:31.092 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-18T22:57:31.092 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-18T22:57:32.808 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-18T22:57:32.809 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-18T22:57:39.692 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-18T22:57:39.805 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T22:57:39.847 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T22:57:39.887 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:57:39.895 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T22:57:39.936 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T22:57:39.989 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T22:57:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T22:57:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T22:57:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T22:57:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T22:57:40.492 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:57:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:57:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:57:40.726 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:57:40.755 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:57:40.791 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T22:57:40.968 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T22:57:41.019 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T22:57:41.048 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:57:41.151 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:57:41.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:57:41.237 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-18T22:57:41.277 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T22:57:41.300 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:57:41.315 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T22:57:41.525 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:57:41.556 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T22:57:41.588 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T22:57:41.693 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T22:57:41.798 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:57:41.859 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-18T22:57:41.922 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:57:41.994 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:57:42.218 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-18T22:57:42.329 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T22:57:42.420 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T22:57:42.457 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T22:57:42.484 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T22:57:42.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T22:57:42.602 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T22:57:42.655 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T22:57:42.694 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T22:57:42.758 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T22:57:42.793 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T22:57:42.837 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T22:57:42.871 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T22:57:42.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T22:57:42.994 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T22:57:43.030 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-18T22:57:43.072 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-18T22:57:43.101 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-18T22:57:43.170 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T22:57:43.215 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T22:57:43.288 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T22:57:43.421 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T22:57:43.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T22:57:43.988 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T22:57:44.062 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T22:57:44.137 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T22:57:44.183 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:57:44.191 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T22:57:44.223 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T22:57:44.264 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T22:57:44.310 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-18T22:57:44.340 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T22:57:44.576 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T22:57:44.724 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T22:57:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T22:57:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T22:57:44.972 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T22:57:45.023 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T22:57:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T22:57:45.142 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T22:57:45.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T22:57:45.307 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T22:57:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T22:57:45.626 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T22:57:45.686 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T22:57:45.736 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T22:57:45.869 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T22:57:45.917 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T22:57:45.947 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T22:57:45.988 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T22:57:46.081 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T22:57:46.122 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T22:57:46.162 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T22:57:46.188 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T22:57:46.212 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T22:57:46.283 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T22:57:46.405 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T22:57:46.502 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T22:57:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T22:57:46.594 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T22:57:46.649 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T22:57:46.832 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T22:57:46.875 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T22:57:47.003 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T22:57:47.048 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T22:57:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T22:57:47.111 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T22:57:47.139 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T22:57:47.173 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T22:57:47.213 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T22:57:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T22:57:47.274 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T22:57:47.350 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T22:57:47.385 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T22:57:47.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T22:57:47.451 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T22:57:47.485 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T22:57:47.574 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T22:57:47.656 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T22:57:47.700 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T22:57:47.731 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T22:57:47.787 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T22:57:47.884 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T22:57:47.933 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T22:57:48.656 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T22:57:48.705 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T22:57:48.739 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T22:57:48.787 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T22:57:48.874 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T22:57:48.907 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T22:57:49.083 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T22:57:49.112 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T22:57:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T22:57:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T22:57:49.943 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T22:57:51.165 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T22:57:51.255 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T22:57:51.311 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T22:57:51.342 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T22:57:51.369 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T22:57:51.682 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T22:57:51.708 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:57:51.798 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-18T22:57:51.829 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-18T22:57:51.857 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T22:57:52.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T22:57:52.344 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:57:52.415 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-18T22:57:52.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-18T22:57:54.663 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:57:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:57:55.242 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-18T22:57:55.280 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:57:55.352 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-18T22:57:55.352 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T22:57:55.352 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T22:57:55.353 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:57:55.381 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:58:06.961 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-18T22:58:06.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-18T22:58:06.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-18T22:58:06.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-18T22:58:06.961 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-18T22:58:06.962 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-18T22:58:06.962 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-18T22:58:06.962 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-18T22:58:06.962 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:07.155 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:58:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-18T22:58:07.170 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:07.170 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T22:58:07.170 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:08.658 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:58:08.683 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-18T22:58:08.683 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:08.683 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T22:58:08.684 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:58:08.684 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T22:58:08.684 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:08.908 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:58:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-18T22:58:08.931 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:08.931 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T22:58:08.931 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:58:08.931 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T22:58:08.931 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:58:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-18T22:58:09.389 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:09.390 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T22:58:09.390 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:58:09.390 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T22:58:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:09.477 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T22:58:09.894 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:58:09.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-18T22:58:09.958 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-18T22:58:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-18T22:58:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:58:10.274 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-18T22:58:10.308 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:58:11.271 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-18T22:58:11.309 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:58:11.409 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-18T22:58:11.570 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-18T22:58:11.602 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T22:58:11.625 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:11.670 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:58:11.680 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-18T22:58:11.725 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-18T22:58:12.602 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T22:58:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:58:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-18T22:58:12.662 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:12.662 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T22:58:12.662 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:58:12.662 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T22:58:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:12.706 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:58:12.728 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-18T22:58:12.728 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:12.728 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T22:58:12.728 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:12.782 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:58:12.803 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:58:12.804 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:12.804 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T22:58:12.804 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:58:12.804 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T22:58:12.804 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:13.115 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T22:58:13.137 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:58:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-18T22:58:17.666 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-18T22:58:17.693 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T22:58:17.873 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-18T22:58:17.985 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T22:58:18.034 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-18T22:58:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T22:58:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T22:58:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T22:58:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-18T22:58:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T22:58:18.271 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-18T22:58:18.317 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-18T22:58:18.369 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T22:58:18.395 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T22:58:18.925 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:58:18.960 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:58:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T22:58:19.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-18T22:58:19.871 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T22:58:19.912 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T22:58:19.993 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T22:58:20.026 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T22:58:20.050 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T22:58:20.076 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T22:58:20.102 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T22:58:20.102 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:58:20.173 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T22:58:20.213 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T22:58:20.214 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:58:20.288 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T22:58:20.314 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T22:58:20.347 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T22:58:20.372 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T22:58:20.406 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T22:58:20.439 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T22:58:20.478 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T22:58:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T22:58:20.536 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T22:58:20.561 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T22:58:20.586 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:58:20.611 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T22:58:20.618 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:58:20.618 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:58:20.797 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T22:58:20.798 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:58:20.872 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T22:58:20.897 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:58:20.923 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T22:58:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T22:58:20.982 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T22:58:21.046 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-18T22:58:25.835 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-18T22:58:25.836 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-18T22:58:25.837 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-18T22:58:25.853 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T22:58:25.854 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T22:58:25.855 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T22:58:25.856 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T22:58:25.857 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T22:58:25.858 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T22:58:25.859 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T22:58:25.860 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T22:58:25.861 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-18T22:58:25.862 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T22:58:25.863 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T22:58:25.864 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T22:58:25.865 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T22:58:28.802 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T22:58:28.802 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:28.802 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-18T22:58:28.802 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.803 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T22:58:28.804 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.805 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.806 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T22:58:28.807 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.808 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T22:58:28.809 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.810 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T22:58:28.811 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T22:58:28.812 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T22:58:28.813 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T22:58:28.814 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T22:58:28.815 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-18T22:58:28.816 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-18T22:58:29.312 DEBUG:teuthology.parallel:result is None 2024-03-18T22:58:29.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:29.460 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:58:29.486 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.486 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.487 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:58:29.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:29.713 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:58:29.737 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.737 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.737 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:58:29.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:29.960 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T22:58:29.987 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-18T22:58:29.988 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-18T22:58:29.989 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T22:58:29.990 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:58:29.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:58:30.024 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:58:30.024 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:58:30.058 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:58:30.059 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T22:58:30.092 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T22:58:30.093 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:58:30.093 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:58:30.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:58:30.193 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:58:30.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:58:30.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:58:30.294 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:58:30.294 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T22:58:30.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T22:58:30.394 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T22:58:30.395 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:58:30.395 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:58:30.425 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:58:30.493 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:58:30.493 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:58:30.525 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:58:30.596 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:58:30.596 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T22:58:30.623 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T22:58:30.690 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T22:58:30.691 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:58:30.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:58:30.718 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:58:30.785 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T22:58:30.785 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:58:30.815 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:58:30.885 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T22:58:30.886 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T22:58:30.913 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T22:58:30.980 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-18T22:58:31.082 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-18T22:58:31.083 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:31.083 INFO:tasks.cephadm:Cluster fsid is 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:58:31.083 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-18T22:58:31.084 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.132', 'mon.b': '172.21.15.144', 'mon.c': '172.21.15.196'} 2024-03-18T22:58:31.084 INFO:tasks.cephadm:First mon is mon.a on smithi132 2024-03-18T22:58:31.084 INFO:tasks.cephadm:First mgr is x 2024-03-18T22:58:31.084 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-18T22:58:31.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2024-03-18T22:58:31.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-03-18T22:58:31.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-03-18T22:58:31.170 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-18T22:58:31.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:31.391 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77736/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2206-gb6ae2491', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2206.gb6ae2491'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:42:27.872142', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240318.181317', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-18T22:58:31.610 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240318.181317, sha1 b6ae24918b03b1b8a19c9263a50655bba80bb810 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:58:31.612 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm 2024-03-18T22:58:31.612 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:32.568 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:58 /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:32.569 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:34.027 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:58 /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:34.028 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:35.422 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 18 22:58 /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:35.423 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:35.441 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:35.460 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T22:58:35.496 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 on all hosts... 2024-03-18T22:58:35.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:58:35.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:58:35.503 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-18T22:58:36.295 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:58:36.305 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:58:36.313 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:59:02.452 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-03-18T22:59:02.453 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-18T22:59:07.229 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T22:59:07.230 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T22:59:09.900 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-18T22:59:09.900 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-18T22:59:09.900 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-18T22:59:09.901 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-03-18T22:59:09.901 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-18T22:59:09.901 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-03-18T22:59:09.901 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-18T22:59:09.925 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2024-03-18T22:59:09.961 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-03-18T22:59:09.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-03-18T22:59:10.026 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2024-03-18T22:59:10.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-03-18T22:59:10.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-03-18T22:59:10.114 INFO:tasks.cephadm:Writing seed config... 2024-03-18T22:59:10.115 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-18T22:59:10.115 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-18T22:59:10.115 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-18T22:59:10.116 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-18T22:59:10.117 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-18T22:59:10.118 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-18T22:59:10.119 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-18T22:59:10.119 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-18T22:59:10.119 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-18T22:59:10.119 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-18T22:59:10.119 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T22:59:10.120 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-18T22:59:10.136 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0a41095a-e57b-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-18T22:59:10.136 DEBUG:teuthology.orchestra.run.smithi132:mon.a> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service 2024-03-18T22:59:10.179 DEBUG:teuthology.orchestra.run.smithi132:mgr.x> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x.service 2024-03-18T22:59:10.221 INFO:tasks.cephadm:Bootstrapping... 2024-03-18T22:59:10.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 -v bootstrap --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.132 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:10.417 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-03-18T22:59:10.417 INFO:teuthology.orchestra.run.smithi132.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810', '-v', 'bootstrap', '--fsid', '0a41095a-e57b-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.132', '--skip-admin-label'] 2024-03-18T22:59:10.442 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:59:10.443 INFO:teuthology.orchestra.run.smithi132.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-18T22:59:10.443 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying podman|docker is present... 2024-03-18T22:59:10.466 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:59:10.466 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying lvm2 is present... 2024-03-18T22:59:10.466 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying time synchronization is in place... 2024-03-18T22:59:10.474 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T22:59:10.474 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T22:59:10.479 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T22:59:10.479 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T22:59:10.485 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout enabled 2024-03-18T22:59:10.491 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout active 2024-03-18T22:59:10.491 INFO:teuthology.orchestra.run.smithi132.stdout:Unit chronyd.service is enabled and running 2024-03-18T22:59:10.491 INFO:teuthology.orchestra.run.smithi132.stdout:Repeating the final host check... 2024-03-18T22:59:10.517 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T22:59:10.517 INFO:teuthology.orchestra.run.smithi132.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-18T22:59:10.517 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl is present 2024-03-18T22:59:10.517 INFO:teuthology.orchestra.run.smithi132.stdout:lvcreate is present 2024-03-18T22:59:10.523 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T22:59:10.523 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T22:59:10.529 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T22:59:10.529 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T22:59:10.535 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout enabled 2024-03-18T22:59:10.541 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout active 2024-03-18T22:59:10.541 INFO:teuthology.orchestra.run.smithi132.stdout:Unit chronyd.service is enabled and running 2024-03-18T22:59:10.541 INFO:teuthology.orchestra.run.smithi132.stdout:Host looks OK 2024-03-18T22:59:10.541 INFO:teuthology.orchestra.run.smithi132.stdout:Cluster fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:10.541 INFO:teuthology.orchestra.run.smithi132.stdout:Acquiring lock 140307023585184 on /run/cephadm/0a41095a-e57b-11ee-95c9-87774f69a715.lock 2024-03-18T22:59:10.542 INFO:teuthology.orchestra.run.smithi132.stdout:Lock 140307023585184 acquired on /run/cephadm/0a41095a-e57b-11ee-95c9-87774f69a715.lock 2024-03-18T22:59:10.542 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying IP 172.21.15.132 port 3300 ... 2024-03-18T22:59:10.542 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying IP 172.21.15.132 port 6789 ... 2024-03-18T22:59:10.542 INFO:teuthology.orchestra.run.smithi132.stdout:Base mon IP(s) is [172.21.15.132:3300, 172.21.15.132:6789], mon addrv is [v2:172.21.15.132:3300,v1:172.21.15.132:6789] 2024-03-18T22:59:10.545 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.132 metric 100 2024-03-18T22:59:10.545 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.132 metric 100 2024-03-18T22:59:10.548 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-18T22:59:10.548 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-18T22:59:10.548 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-03-18T22:59:10.550 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-18T22:59:10.550 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-18T22:59:10.550 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:Mon IP `172.21.15.132` is in CIDR network `172.21.0.0/20` 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:Mon IP `172.21.15.132` is in CIDR network `172.21.0.0/20` 2024-03-18T22:59:10.551 INFO:teuthology.orchestra.run.smithi132.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-18T22:59:10.552 INFO:teuthology.orchestra.run.smithi132.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-18T22:59:10.552 INFO:teuthology.orchestra.run.smithi132.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:59:11.046 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stdout 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-18T22:59:11.046 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-18T22:59:11.047 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Getting image source signatures 2024-03-18T22:59:11.047 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying blob sha256:93b5a8d362aeee7e8b1f0bbf18238e1bb256c50573ad5b72aff9e382b1bb3ccb 2024-03-18T22:59:11.047 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-18T22:59:11.047 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Copying config sha256:2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-18T22:59:11.047 INFO:teuthology.orchestra.run.smithi132.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-18T22:59:11.993 INFO:teuthology.orchestra.run.smithi132.stdout:ceph: stdout ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-18T22:59:11.993 INFO:teuthology.orchestra.run.smithi132.stdout:Ceph version: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-18T22:59:11.993 INFO:teuthology.orchestra.run.smithi132.stdout:Extracting ceph user uid/gid from container image... 2024-03-18T22:59:12.916 INFO:teuthology.orchestra.run.smithi132.stdout:stat: stdout 167 167 2024-03-18T22:59:12.916 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial keys... 2024-03-18T22:59:13.830 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQBBx/hl7GvwDhAApkI0+pjKHsB/I1cDoIo2PA== 2024-03-18T22:59:14.760 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQBCx/hlEa0cChAAq6d0I2oz0ozi6wRvVpF8cg== 2024-03-18T22:59:15.659 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph-authtool: stdout AQBDx/hlA3IhBBAAg6FhobZ6tNFUEjEUiOTygQ== 2024-03-18T22:59:15.660 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial monmap... 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:monmaptool for a [v2:172.21.15.132:3300,v1:172.21.15.132:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:setting min_mon_release = quincy 2024-03-18T22:59:16.614 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: set fsid to 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:16.615 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T22:59:16.615 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T22:59:16.615 INFO:teuthology.orchestra.run.smithi132.stdout:Creating mon... 2024-03-18T22:59:17.587 INFO:teuthology.orchestra.run.smithi132.stdout:create mon.a on 2024-03-18T22:59:17.790 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-18T22:59:17.928 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-18T22:59:18.095 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target. 2024-03-18T22:59:18.095 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target. 2024-03-18T22:59:18.320 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a 2024-03-18T22:59:18.321 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-18T22:59:18.487 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target.wants/ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715@.service. 2024-03-18T22:59:19.053 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:59:19.053 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-03-18T22:59:19.058 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:59:19.059 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T22:59:19.059 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-03-18T22:59:19.059 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T22:59:19.061 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mon to start... 2024-03-18T22:59:19.062 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mon... 2024-03-18T22:59:20.106 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout cluster: 2024-03-18T22:59:20.111 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout id: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:20.111 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout services: 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.519972s) 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout data: 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout pgs: 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:20.112 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.481+0000 7f7b4a45e700 1 Processor -- start 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.481+0000 7f7b4a45e700 1 -- start start 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.481+0000 7f7b4a45e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.481+0000 7f7b4a45e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44108760 con 0x7f7b44107e00 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44146/0 (socket says 172.21.15.132:44146) 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b43fff700 1 -- 172.21.15.132:0/2862098464 learned_addr learned my addr 172.21.15.132:0/2862098464 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b43fff700 1 -- 172.21.15.132:0/2862098464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b44108f60 con 0x7f7b44107e00 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b43fff700 1 --2- 172.21.15.132:0/2862098464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b340098a0 tx=0x7f7b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894dbbeff7a67bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b42ffd700 1 -- 172.21.15.132:0/2862098464 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b34004160 con 0x7f7b44107e00 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b42ffd700 1 -- 172.21.15.132:0/2862098464 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34004300 con 0x7f7b44107e00 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.482+0000 7f7b42ffd700 1 -- 172.21.15.132:0/2862098464 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b340036a0 con 0x7f7b44107e00 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 -- 172.21.15.132:0/2862098464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 msgr2=0x7f7b44108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 --2- 172.21.15.132:0/2862098464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b340098a0 tx=0x7f7b34009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 -- 172.21.15.132:0/2862098464 shutdown_connections 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 --2- 172.21.15.132:0/2862098464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b44108220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 -- 172.21.15.132:0/2862098464 >> 172.21.15.132:0/2862098464 conn(0x7f7b44103380 msgr2=0x7f7b441057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 -- 172.21.15.132:0/2862098464 shutdown_connections 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.483+0000 7f7b4a45e700 1 -- 172.21.15.132:0/2862098464 wait complete. 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b4a45e700 1 Processor -- start 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b4a45e700 1 -- start start 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b4a45e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b4a45e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44109430 con 0x7f7b44107e00 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44158/0 (socket says 172.21.15.132:44158) 2024-03-18T22:59:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b43fff700 1 -- 172.21.15.132:0/1622192946 learned_addr learned my addr 172.21.15.132:0/1622192946 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b43fff700 1 -- 172.21.15.132:0/1622192946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4419dc70 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.484+0000 7f7b43fff700 1 --2- 172.21.15.132:0/1622192946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b34000c00 tx=0x7f7b34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b34003db0 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b34011630 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4419df00 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b34011950 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441a0bf0 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b3401a460 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.485+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b3401b780 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.486+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b440693f0 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.488+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7b3401ba20 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.558+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7b441a0dc0 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.558+0000 7f7b417fa700 1 -- 172.21.15.132:0/1622192946 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7b3401bc00 con 0x7f7b44107e00 2024-03-18T22:59:20.115 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 msgr2=0x7f7b4419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 --2- 172.21.15.132:0/1622192946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b34000c00 tx=0x7f7b34004750 comp rx=0 tx=0).stop 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 shutdown_connections 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 --2- 172.21.15.132:0/1622192946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b44107e00 0x7f7b4419d730 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 >> 172.21.15.132:0/1622192946 conn(0x7f7b44103380 msgr2=0x7f7b4407d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 shutdown_connections 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:19.559+0000 7f7b4a45e700 1 -- 172.21.15.132:0/1622192946 wait complete. 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:mon is available 2024-03-18T22:59:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:Assimilating anything we can from ceph.conf... 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout fsid = 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.132:3300,v1:172.21.15.132:6789] 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T22:59:21.146 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.544+0000 7f5330617700 1 Processor -- start 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.544+0000 7f5330617700 1 -- start start 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f5330617700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f5330617700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f532807c880 con 0x7f532807de50 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f532e3b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:21.147 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f532e3b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44162/0 (socket says 172.21.15.132:44162) 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f532e3b3700 1 -- 172.21.15.132:0/1098004922 learned_addr learned my addr 172.21.15.132:0/1098004922 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f532e3b3700 1 -- 172.21.15.132:0/1098004922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532807ca00 con 0x7f532807de50 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.545+0000 7f532e3b3700 1 --2- 172.21.15.132:0/1098004922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53240098a0 tx=0x7f5324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567b4eb098ea6fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f532d3b1700 1 -- 172.21.15.132:0/1098004922 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5324004160 con 0x7f532807de50 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f532d3b1700 1 -- 172.21.15.132:0/1098004922 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5324004300 con 0x7f532807de50 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f532d3b1700 1 -- 172.21.15.132:0/1098004922 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53240036a0 con 0x7f532807de50 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 -- 172.21.15.132:0/1098004922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 msgr2=0x7f532807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 --2- 172.21.15.132:0/1098004922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53240098a0 tx=0x7f5324009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 -- 172.21.15.132:0/1098004922 shutdown_connections 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 --2- 172.21.15.132:0/1098004922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532807c2b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 -- 172.21.15.132:0/1098004922 >> 172.21.15.132:0/1098004922 conn(0x7f5328103380 msgr2=0x7f53281057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 -- 172.21.15.132:0/1098004922 shutdown_connections 2024-03-18T22:59:21.148 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.546+0000 7f5330617700 1 -- 172.21.15.132:0/1098004922 wait complete. 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f5330617700 1 Processor -- start 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f5330617700 1 -- start start 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f5330617700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f5330617700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f532807cf30 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f532e3b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f532e3b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44174/0 (socket says 172.21.15.132:44174) 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.547+0000 7f532e3b3700 1 -- 172.21.15.132:0/1637028771 learned_addr learned my addr 172.21.15.132:0/1637028771 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f532e3b3700 1 -- 172.21.15.132:0/1637028771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532819dc00 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f532e3b3700 1 --2- 172.21.15.132:0/1637028771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5324000c00 tx=0x7f5324004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5324003db0 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5324011630 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532819de90 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5324011950 con 0x7f532807de50 2024-03-18T22:59:21.149 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53281a0b80 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5324011af0 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.548+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f532401c790 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.549+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53280693f0 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.551+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5324004260 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.619+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f532802dcb0 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.622+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+458 (secure 0 0 0) 0x7f532401c990 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.623+0000 7f531f7fe700 1 -- 172.21.15.132:0/1637028771 <== mon.0 v2:172.21.15.132:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f532401b910 con 0x7f532807de50 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 msgr2=0x7f532819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 --2- 172.21.15.132:0/1637028771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5324000c00 tx=0x7f5324004750 comp rx=0 tx=0).stop 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 shutdown_connections 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 --2- 172.21.15.132:0/1637028771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f532807de50 0x7f532819d6c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 >> 172.21.15.132:0/1637028771 conn(0x7f5328103380 msgr2=0x7f53281056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:21.150 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 shutdown_connections 2024-03-18T22:59:21.151 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:20.624+0000 7f5330617700 1 -- 172.21.15.132:0/1637028771 wait complete. 2024-03-18T22:59:21.151 INFO:teuthology.orchestra.run.smithi132.stdout:Generating new minimal ceph.conf... 2024-03-18T22:59:22.233 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 ceph-mon[31645]: from='client.? 172.21.15.132:0/1542897376' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T22:59:22.238 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7dfcbb5700 1 Processor -- start 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7dfcbb5700 1 -- start start 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7dfcbb5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7dfcbb5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8108790 con 0x7f7df8107e30 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7df659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7df659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44980/0 (socket says 172.21.15.132:44980) 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.551+0000 7f7df659c700 1 -- 172.21.15.132:0/3007450544 learned_addr learned my addr 172.21.15.132:0/3007450544 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7df659c700 1 -- 172.21.15.132:0/3007450544 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8108f90 con 0x7f7df8107e30 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7df659c700 1 --2- 172.21.15.132:0/3007450544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7de80098a0 tx=0x7f7de8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e12dff6da465d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7df559a700 1 -- 172.21.15.132:0/3007450544 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de8004160 con 0x7f7df8107e30 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7df559a700 1 -- 172.21.15.132:0/3007450544 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7de8004300 con 0x7f7df8107e30 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7df559a700 1 -- 172.21.15.132:0/3007450544 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de8011b50 con 0x7f7df8107e30 2024-03-18T22:59:22.239 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/3007450544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 msgr2=0x7f7df8108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.552+0000 7f7dfcbb5700 1 --2- 172.21.15.132:0/3007450544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7de80098a0 tx=0x7f7de8009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/3007450544 shutdown_connections 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 --2- 172.21.15.132:0/3007450544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df8108250 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/3007450544 >> 172.21.15.132:0/3007450544 conn(0x7f7df81033d0 msgr2=0x7f7df8105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/3007450544 shutdown_connections 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/3007450544 wait complete. 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 Processor -- start 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- start start 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.553+0000 7f7dfcbb5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7df810c250 con 0x7f7df8107e30 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7df659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7df659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44996/0 (socket says 172.21.15.132:44996) 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7df659c700 1 -- 172.21.15.132:0/1542897376 learned_addr learned my addr 172.21.15.132:0/1542897376 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7df659c700 1 -- 172.21.15.132:0/1542897376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df819c270 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7df659c700 1 --2- 172.21.15.132:0/1542897376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011670 tx=0x7f7de8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de80039c0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7de8011e50 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de801a5c0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df819c500 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df819f1f0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7de801ac00 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.554+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7de8019700 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.555+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df810c3d0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.557+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7de8003b60 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.624+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7df80693f0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.624+0000 7f7de77fe700 1 -- 172.21.15.132:0/1542897376 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f7de80199a0 con 0x7f7df8107e30 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 msgr2=0x7f7df819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:22.241 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 --2- 172.21.15.132:0/1542897376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011670 tx=0x7f7de8011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 shutdown_connections 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 --2- 172.21.15.132:0/1542897376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7df8107e30 0x7f7df819bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 >> 172.21.15.132:0/1542897376 conn(0x7f7df81033d0 msgr2=0x7f7df8103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 shutdown_connections 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:21.625+0000 7f7dfcbb5700 1 -- 172.21.15.132:0/1542897376 wait complete. 2024-03-18T22:59:22.242 INFO:teuthology.orchestra.run.smithi132.stdout:Restarting the monitor... 2024-03-18T22:59:22.514 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 systemd[1]: Stopping Ceph mon.a for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T22:59:22.827 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a[31641]: 2024-03-18T22:59:22.512+0000 7faf0d8a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T22:59:22.827 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a[31641]: 2024-03-18T22:59:22.512+0000 7faf0d8a0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-18T22:59:22.827 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 podman[31891]: 2024-03-18 22:59:22.605121617 +0000 UTC m=+0.210068251 container died 06a89495f66c6eaf46a9183c791058b3e37346b13901ef9ba462502dde8e3324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T22:59:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 podman[31891]: 2024-03-18 22:59:22.901829073 +0000 UTC m=+0.506775706 container cleanup 06a89495f66c6eaf46a9183c791058b3e37346b13901ef9ba462502dde8e3324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-18T22:59:23.111 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:22 smithi132 bash[31891]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a 2024-03-18T22:59:23.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 podman[31905]: 2024-03-18 22:59:23.111185732 +0000 UTC m=+0.504130592 container remove 06a89495f66c6eaf46a9183c791058b3e37346b13901ef9ba462502dde8e3324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-18T22:59:23.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-18T22:59:23.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 systemd[1]: Stopped Ceph mon.a for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T22:59:23.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 systemd[1]: Starting Ceph mon.a for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T22:59:23.689 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 podman[32001]: 2024-03-18 22:59:23.412046208 +0000 UTC m=+0.106172246 container create b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-18T22:59:23.689 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 podman[32001]: 2024-03-18 22:59:23.324166684 +0000 UTC m=+0.018292738 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:59:23.689 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 podman[32001]: 2024-03-18 22:59:23.614517755 +0000 UTC m=+0.308643800 container init b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-18T22:59:23.689 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 podman[32001]: 2024-03-18 22:59:23.617605941 +0000 UTC m=+0.311731975 container start b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-18T22:59:23.689 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: pidfile_write: ignore empty --pid-file 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: load: jerasure load: lrc 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: RocksDB version: 7.9.2 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Git sha 0 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: DB SUMMARY 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: DB Session ID: JVBMPDC3AM1XVYFGK9JG 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: CURRENT file: CURRENT 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104572 ; 2024-03-18T22:59:23.690 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.error_if_exists: 0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.create_if_missing: 0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.paranoid_checks: 1 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.env: 0x5604515acc60 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.info_log: 0x560453cfdae0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.statistics: (nil) 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.use_fsync: 0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_log_file_size: 0 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T22:59:23.691 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_fallocate: 1 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.use_direct_reads: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.db_log_dir: 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.wal_dir: 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T22:59:23.692 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.write_buffer_manager: 0x560453cd01e0 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T22:59:23.693 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T22:59:23.694 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T22:59:23.694 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.unordered_write: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.row_cache: None 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.wal_filter: None 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.two_write_queues: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.wal_compression: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.atomic_flush: 0 2024-03-18T22:59:23.695 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.log_readahead_size: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_background_jobs: 2 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_background_compactions: -1 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_subcompactions: 1 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T22:59:23.696 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_open_files: -1 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_background_flushes: -1 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Compression algorithms supported: 2024-03-18T22:59:23.697 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kZSTD supported: 0 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kXpressCompression supported: 0 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kZlibCompression supported: 1 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kSnappyCompression supported: 1 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kLZ4Compression supported: 1 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: kBZip2Compression supported: 0 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T22:59:23.698 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.merge_operator: 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_filter: None 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560453cfdbe0) 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: cache_index_and_filter_blocks: 1 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: pin_top_level_index_and_filter: 1 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_type: 0 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: data_block_index_type: 0 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_shortening: 1 2024-03-18T22:59:23.699 INFO:journalctl@ceph.mon.a.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: checksum: 4 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: no_block_cache: 0 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache: 0x560453d03090 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_name: BinnedLRUCache 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_options: 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: capacity : 536870912 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: num_shard_bits : 4 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: strict_capacity_limit : 0 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: high_pri_pool_ratio: 0.000 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_cache_compressed: (nil) 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: persistent_cache: (nil) 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_size: 4096 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_size_deviation: 10 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_restart_interval: 16 2024-03-18T22:59:23.700 INFO:journalctl@ceph.mon.a.smithi132.stdout: index_block_restart_interval: 1 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: metadata_block_size: 4096 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: partition_filters: 0 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: use_delta_encoding: 1 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: filter_policy: bloomfilter 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: whole_key_filtering: 1 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: verify_compression: 0 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: read_amp_bytes_per_bit: 0 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: format_version: 5 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: enable_index_compression: 1 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: block_align: 0 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: max_auto_readahead_size: 262144 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: prepopulate_block_cache: 0 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: initial_auto_readahead_size: 8192 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T22:59:23.701 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression: NoCompression 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.num_levels: 7 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T22:59:23.702 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T22:59:23.703 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T22:59:23.704 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T22:59:23.705 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.table_properties_collectors: 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.inplace_update_support: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.bloom_locality: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.max_successive_merges: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T22:59:23.706 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.ttl: 2592000 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enable_blob_files: false 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.min_blob_size: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T22:59:23.707 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 86481e10-060c-4c62-ab68-16fee260e5c3 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802763646011, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802763647009, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97699, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14048, "raw_average_key_size": 49, "raw_value_size": 90019, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802763, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "86481e10-060c-4c62-ab68-16fee260e5c3", "db_session_id": "JVBMPDC3AM1XVYFGK9JG", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802763647108, "job": 1, "event": "recovery_finished"} 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560453df6000 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: DB pointer 0x560453de0000 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** DB Stats ** 2024-03-18T22:59:23.708 INFO:journalctl@ceph.mon.a.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.709 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** Compaction Stats [default] ** 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T22:59:23.710 INFO:journalctl@ceph.mon.a.smithi132.stdout: Cumulative compaction: 0.00 GB write, 32.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: Interval compaction: 0.00 GB write, 32.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: Block cache BinnedLRUCache@0x560453d03090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: starting mon.a rank 0 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???) e1 preinit fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).mds e1 new map 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).mds e1 print_map 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: e1 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: legacy client fscid: -1 2024-03-18T22:59:23.711 INFO:journalctl@ceph.mon.a.smithi132.stdout: 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout: No filesystems configured 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 bash[32001]: b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 2024-03-18T22:59:23.712 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 systemd[1]: Started Ceph mon.a for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T22:59:23.712 INFO:teuthology.orchestra.run.smithi132.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: monmap epoch 1 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: last_changed 2024-03-18T22:59:16.031249+0000 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: min_mon_release 19 (squid) 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: election_strategy: 1 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: fsmap 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T22:59:23.948 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:23 smithi132 ceph-mon[32020]: mgrmap e1: no daemons active 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d92a47700 1 Processor -- start 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d92a47700 1 -- start start 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d92a47700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d92a47700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c106ac0 con 0x7f6d8c1060d0 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45010/0 (socket says 172.21.15.132:45010) 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.160+0000 7f6d8bfff700 1 -- 172.21.15.132:0/2946210446 learned_addr learned my addr 172.21.15.132:0/2946210446 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.161+0000 7f6d8bfff700 1 -- 172.21.15.132:0/2946210446 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c109060 con 0x7f6d8c1060d0 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.161+0000 7f6d8bfff700 1 --2- 172.21.15.132:0/2946210446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29f83c5c28d2da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.161+0000 7f6d8affd700 1 -- 172.21.15.132:0/2946210446 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c004160 con 0x7f6d8c1060d0 2024-03-18T22:59:24.811 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.161+0000 7f6d8affd700 1 -- 172.21.15.132:0/2946210446 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d7c004300 con 0x7f6d8c1060d0 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.161+0000 7f6d8affd700 1 -- 172.21.15.132:0/2946210446 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c011b50 con 0x7f6d8c1060d0 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- 172.21.15.132:0/2946210446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 msgr2=0x7f6d8c1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 --2- 172.21.15.132:0/2946210446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0098a0 tx=0x7f6d7c009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- 172.21.15.132:0/2946210446 shutdown_connections 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 --2- 172.21.15.132:0/2946210446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c1064f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- 172.21.15.132:0/2946210446 >> 172.21.15.132:0/2946210446 conn(0x7f6d8c101a20 msgr2=0x7f6d8c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- 172.21.15.132:0/2946210446 shutdown_connections 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- 172.21.15.132:0/2946210446 wait complete. 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 Processor -- start 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.162+0000 7f6d92a47700 1 -- start start 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d92a47700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d92a47700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10c410 con 0x7f6d8c1060d0 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:24.812 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45024/0 (socket says 172.21.15.132:45024) 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d8bfff700 1 -- 172.21.15.132:0/2752604290 learned_addr learned my addr 172.21.15.132:0/2752604290 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d8bfff700 1 -- 172.21.15.132:0/2752604290 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c108dd0 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d8bfff700 1 --2- 172.21.15.132:0/2752604290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000c00 tx=0x7f6d7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c003b20 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d7c003cc0 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c107010 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c024910 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.163+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1074f0 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.164+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d7c022020 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.164+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6d7c011e00 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.164+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c063940 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.167+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6d7c01acd0 con 0x7f6d8c1060d0 2024-03-18T22:59:24.813 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.233+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f6d8c107770 con 0x7f6d8c1060d0 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d897fa700 1 -- 172.21.15.132:0/2752604290 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6d7c032030 con 0x7f6d8c1060d0 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 msgr2=0x7f6d8c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 --2- 172.21.15.132:0/2752604290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000c00 tx=0x7f6d7c011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 shutdown_connections 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 --2- 172.21.15.132:0/2752604290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c1060d0 0x7f6d8c108890 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 >> 172.21.15.132:0/2752604290 conn(0x7f6d8c101a20 msgr2=0x7f6d8c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.235+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 shutdown_connections 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:24.236+0000 7f6d92a47700 1 -- 172.21.15.132:0/2752604290 wait complete. 2024-03-18T22:59:24.814 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-18T22:59:24.836 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-18T22:59:24.837 INFO:teuthology.orchestra.run.smithi132.stdout:Creating mgr... 2024-03-18T22:59:24.837 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-18T22:59:24.837 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-18T22:59:25.093 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x 2024-03-18T22:59:25.093 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-18T22:59:25.236 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715.target.wants/ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-0a41095a-e57b-11ee-95c9-87774f69a715@.service. 2024-03-18T22:59:25.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:25 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2752604290' entity='client.admin' 2024-03-18T22:59:25.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 systemd[1]: Starting Ceph mgr.x for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T22:59:25.725 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 podman[32238]: 2024-03-18 22:59:25.464902535 +0000 UTC m=+0.109446704 container create a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T22:59:25.725 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 podman[32238]: 2024-03-18 22:59:25.373259338 +0000 UTC m=+0.017803523 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-18T22:59:25.725 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 podman[32238]: 2024-03-18 22:59:25.650422376 +0000 UTC m=+0.294966559 container init a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-18T22:59:25.725 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 podman[32238]: 2024-03-18 22:59:25.652835545 +0000 UTC m=+0.297379716 container start a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T22:59:25.725 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 bash[32238]: a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c 2024-03-18T22:59:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:59:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-03-18T22:59:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:59:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T22:59:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-03-18T22:59:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T22:59:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T22:59:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr to start... 2024-03-18T22:59:25.768 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr... 2024-03-18T22:59:25.998 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:25 smithi132 systemd[1]: Started Ceph mgr.x for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T22:59:26.265 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:26.214+0000 7f2fb69c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T22:59:26.564 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:26 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2345372435' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:59:26.564 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:26.310+0000 7f2fb69c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:59:26.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:59:26.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:59:26.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:59:19.052412+0000", 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:59:26.848 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.186+0000 7fede937b700 1 Processor -- start 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.186+0000 7fede937b700 1 -- start start 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.186+0000 7fede937b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.186+0000 7fede937b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fede407bde0 con 0x7fede41088f0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45052/0 (socket says 172.21.15.132:45052) 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede2ffd700 1 -- 172.21.15.132:0/2982124502 learned_addr learned my addr 172.21.15.132:0/2982124502 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede2ffd700 1 -- 172.21.15.132:0/2982124502 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede410b220 con 0x7fede41088f0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede2ffd700 1 --2- 172.21.15.132:0/2982124502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e80 tx=0x7feddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a267823423784 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede1ffb700 1 -- 172.21.15.132:0/2982124502 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feddc017070 con 0x7fede41088f0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede1ffb700 1 -- 172.21.15.132:0/2982124502 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feddc00b8c0 con 0x7fede41088f0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.187+0000 7fede1ffb700 1 -- 172.21.15.132:0/2982124502 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feddc0045c0 con 0x7fede41088f0 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 -- 172.21.15.132:0/2982124502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 msgr2=0x7fede410ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:26.849 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 --2- 172.21.15.132:0/2982124502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e80 tx=0x7feddc00b330 comp rx=0 tx=0).stop 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 -- 172.21.15.132:0/2982124502 shutdown_connections 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 --2- 172.21.15.132:0/2982124502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede410ace0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 -- 172.21.15.132:0/2982124502 >> 172.21.15.132:0/2982124502 conn(0x7fede41022e0 msgr2=0x7fede4104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.188+0000 7fede937b700 1 -- 172.21.15.132:0/2982124502 shutdown_connections 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.189+0000 7fede937b700 1 -- 172.21.15.132:0/2982124502 wait complete. 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.189+0000 7fede937b700 1 Processor -- start 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.189+0000 7fede937b700 1 -- start start 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.189+0000 7fede937b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.189+0000 7fede937b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fede410c1e0 con 0x7fede41088f0 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45056/0 (socket says 172.21.15.132:45056) 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede2ffd700 1 -- 172.21.15.132:0/2345372435 learned_addr learned my addr 172.21.15.132:0/2345372435 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede2ffd700 1 -- 172.21.15.132:0/2345372435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede41086b0 con 0x7fede41088f0 2024-03-18T22:59:26.850 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede2ffd700 1 --2- 172.21.15.132:0/2345372435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feddc0072a0 tx=0x7feddc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feddc020020 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feddc003b50 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede41068a0 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feddc017040 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede4106de0 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.190+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feddc023020 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.191+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feddc00f050 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.191+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede40693f0 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.193+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feddc02a020 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.264+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fede4106fb0 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.265+0000 7feddbfff700 1 -- 172.21.15.132:0/2345372435 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7feddc00b860 con 0x7fede41088f0 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 msgr2=0x7fede4108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 --2- 172.21.15.132:0/2345372435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feddc0072a0 tx=0x7feddc00bd50 comp rx=0 tx=0).stop 2024-03-18T22:59:26.851 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 shutdown_connections 2024-03-18T22:59:26.852 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 --2- 172.21.15.132:0/2345372435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede41088f0 0x7fede4108170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:26.852 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 >> 172.21.15.132:0/2345372435 conn(0x7fede41022e0 msgr2=0x7fede4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:26.852 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 shutdown_connections 2024-03-18T22:59:26.852 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:26.266+0000 7fede937b700 1 -- 172.21.15.132:0/2345372435 wait complete. 2024-03-18T22:59:26.852 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (1/15)... 2024-03-18T22:59:27.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:26.958+0000 7f2fb69c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T22:59:27.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.050+0000 7f2fb69c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T22:59:27.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.192+0000 7f2fb69c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T22:59:27.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.285+0000 7f2fb69c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T22:59:27.616 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.368+0000 7f2fb69c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T22:59:27.915 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.629+0000 7f2fb69c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T22:59:27.916 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.737+0000 7f2fb69c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T22:59:27.916 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.820+0000 7f2fb69c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T22:59:27.916 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:27.915+0000 7f2fb69c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T22:59:28.365 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:28 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:28.089+0000 7f2fb69c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T22:59:29.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:29.059+0000 7f2fb69c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T22:59:29.446 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:29.194+0000 7f2fb69c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T22:59:29.833 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4266490264' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:59:29.833 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:29.548+0000 7f2fb69c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T22:59:29.833 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:29.636+0000 7f2fb69c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T22:59:29.833 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:29.832+0000 7f2fb69c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:59:30.035 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:30.035 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:30.035 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T22:59:30.035 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:59:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:59:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:59:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:59:30.045 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.046 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:59:19.052412+0000", 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:30.047 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.314+0000 7faac7e95700 1 Processor -- start 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac7e95700 1 -- start start 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac7e95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac7e95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faac007be20 con 0x7faac01088a0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac5c31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac5c31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45066/0 (socket says 172.21.15.132:45066) 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac5c31700 1 -- 172.21.15.132:0/99346530 learned_addr learned my addr 172.21.15.132:0/99346530 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac5c31700 1 -- 172.21.15.132:0/99346530 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac010b1d0 con 0x7faac01088a0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.315+0000 7faac5c31700 1 --2- 172.21.15.132:0/99346530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faabc0098a0 tx=0x7faabc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a129bc86f18c04e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac4c2f700 1 -- 172.21.15.132:0/99346530 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faabc004160 con 0x7faac01088a0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac4c2f700 1 -- 172.21.15.132:0/99346530 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faabc004300 con 0x7faac01088a0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac4c2f700 1 -- 172.21.15.132:0/99346530 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faabc011b50 con 0x7faac01088a0 2024-03-18T22:59:30.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 -- 172.21.15.132:0/99346530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 msgr2=0x7faac010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 --2- 172.21.15.132:0/99346530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faabc0098a0 tx=0x7faabc009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 -- 172.21.15.132:0/99346530 shutdown_connections 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 --2- 172.21.15.132:0/99346530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac010ac90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 -- 172.21.15.132:0/99346530 >> 172.21.15.132:0/99346530 conn(0x7faac0102290 msgr2=0x7faac01046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 -- 172.21.15.132:0/99346530 shutdown_connections 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.316+0000 7faac7e95700 1 -- 172.21.15.132:0/99346530 wait complete. 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac7e95700 1 Processor -- start 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac7e95700 1 -- start start 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac7e95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac7e95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faac010c1f0 con 0x7faac01088a0 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac5c31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac5c31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45068/0 (socket says 172.21.15.132:45068) 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac5c31700 1 -- 172.21.15.132:0/4266490264 learned_addr learned my addr 172.21.15.132:0/4266490264 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:30.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.317+0000 7faac5c31700 1 -- 172.21.15.132:0/4266490264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0108610 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faac5c31700 1 --2- 172.21.15.132:0/4266490264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faabc011670 tx=0x7faabc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faabc0039c0 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faabc011e50 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac0106850 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac0106d30 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.318+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faabc01a5c0 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.319+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faabc01ac50 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.319+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faabc019700 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.319+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa4005360 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.321+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7faabc003b60 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.394+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7faaa4005920 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.395+0000 7faab6ffd700 1 -- 172.21.15.132:0/4266490264 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7faabc024430 con 0x7faac01088a0 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 msgr2=0x7faac01080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:30.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 --2- 172.21.15.132:0/4266490264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faabc011670 tx=0x7faabc011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 shutdown_connections 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 --2- 172.21.15.132:0/4266490264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faac01088a0 0x7faac01080d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 >> 172.21.15.132:0/4266490264 conn(0x7faac0102290 msgr2=0x7faac0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 shutdown_connections 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:29.396+0000 7faac7e95700 1 -- 172.21.15.132:0/4266490264 wait complete. 2024-03-18T22:59:30.051 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (2/15)... 2024-03-18T22:59:30.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:30.020+0000 7f2fb69c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T22:59:30.569 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:30.309+0000 7f2fb69c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:59:30.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:30.569+0000 7f2fb69c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T22:59:30.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:30.804+0000 7f2fb69c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T22:59:31.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:31 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:31.598+0000 7f2fb69c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T22:59:32.449 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:32.173+0000 7f2fb69c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T22:59:32.449 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:32.271+0000 7f2fb69c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T22:59:32.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:32.495+0000 7f2fb69c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: Activating manager daemon x 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: mgrmap e2: x(active, starting, since 0.00328242s) 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: Manager daemon x is now available 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' 2024-03-18T22:59:32.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T22:59:32.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' 2024-03-18T22:59:32.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='mgr.14100 172.21.15.132:0/4163212346' entity='mgr.x' 2024-03-18T22:59:32.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:32 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1027888587' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:59:34.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:34 smithi132 ceph-mon[32020]: mgrmap e3: x(active, since 1.00758s) 2024-03-18T22:59:36.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:36 smithi132 ceph-mon[32020]: mgrmap e4: x(active, since 3s) 2024-03-18T22:59:36.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:36.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:36.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T22:59:36.913 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-18T22:59:36.914 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:59:36.915 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:59:36.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:59:36.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:59:36.923 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:59:36.924 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:36.925 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:59:19.052412+0000", 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.471+0000 7f964355a700 1 Processor -- start 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f964355a700 1 -- start start 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f964355a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f964355a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f963c109db0 con 0x7f963c1093c0 2024-03-18T22:59:36.926 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f96412f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f96412f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34394/0 (socket says 172.21.15.132:34394) 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f96412f6700 1 -- 172.21.15.132:0/2964784527 learned_addr learned my addr 172.21.15.132:0/2964784527 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f96412f6700 1 -- 172.21.15.132:0/2964784527 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c109f30 con 0x7f963c1093c0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.472+0000 7f96412f6700 1 --2- 172.21.15.132:0/2964784527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f9638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30f2cbcc64b0e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f9633fff700 1 -- 172.21.15.132:0/2964784527 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9638004160 con 0x7f963c1093c0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f9633fff700 1 -- 172.21.15.132:0/2964784527 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9638004300 con 0x7f963c1093c0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f9633fff700 1 -- 172.21.15.132:0/2964784527 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9638011b50 con 0x7f963c1093c0 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 -- 172.21.15.132:0/2964784527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 msgr2=0x7f963c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 --2- 172.21.15.132:0/2964784527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96380098a0 tx=0x7f9638009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 -- 172.21.15.132:0/2964784527 shutdown_connections 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 --2- 172.21.15.132:0/2964784527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c1097e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 -- 172.21.15.132:0/2964784527 >> 172.21.15.132:0/2964784527 conn(0x7f963c101a00 msgr2=0x7f963c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:36.927 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 -- 172.21.15.132:0/2964784527 shutdown_connections 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.473+0000 7f964355a700 1 -- 172.21.15.132:0/2964784527 wait complete. 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f964355a700 1 Processor -- start 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f964355a700 1 -- start start 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f964355a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f964355a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f963c106c20 con 0x7f963c1093c0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f96412f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f96412f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34396/0 (socket says 172.21.15.132:34396) 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f96412f6700 1 -- 172.21.15.132:0/1027888587 learned_addr learned my addr 172.21.15.132:0/1027888587 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f96412f6700 1 -- 172.21.15.132:0/1027888587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19c2a0 con 0x7f963c1093c0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.474+0000 7f96412f6700 1 --2- 172.21.15.132:0/1027888587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9638011670 tx=0x7f9638011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96380039c0 con 0x7f963c1093c0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9638011e50 con 0x7f963c1093c0 2024-03-18T22:59:36.928 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19c590 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f963801a5c0 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c19f220 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f963801ac50 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.475+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9638019700 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.476+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c0693f0 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.478+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9638032080 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.511+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 2) ==== 45862+0+0 (secure 0 0 0) 0x7f96380199a0 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.557+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f963c19f7f0 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.557+0000 7f96327fc700 1 -- 172.21.15.132:0/1027888587 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f963800b3e0 con 0x7f963c1093c0 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 msgr2=0x7f963c19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 --2- 172.21.15.132:0/1027888587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9638011670 tx=0x7f9638011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 shutdown_connections 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 --2- 172.21.15.132:0/1027888587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1093c0 0x7f963c19bd60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:36.929 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 >> 172.21.15.132:0/1027888587 conn(0x7f963c101a00 msgr2=0x7f963c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:36.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 shutdown_connections 2024-03-18T22:59:36.930 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:32.558+0000 7f964355a700 1 -- 172.21.15.132:0/1027888587 wait complete. 2024-03-18T22:59:36.930 INFO:teuthology.orchestra.run.smithi132.stdout:mgr not available, waiting (3/15)... 2024-03-18T22:59:43.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:43 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1580799298' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T22:59:43.887 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 0 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T22:59:43.888 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T22:59:43.889 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T22:59:43.890 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ], 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T22:59:19.052412+0000", 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T22:59:43.891 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout }, 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd87558f700 1 Processor -- start 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd87558f700 1 -- start start 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd87558f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd87558f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd870107760 con 0x7fd870106d70 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd86effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd86effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43702/0 (socket says 172.21.15.132:43702) 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.054+0000 7fd86effd700 1 -- 172.21.15.132:0/4225528601 learned_addr learned my addr 172.21.15.132:0/4225528601 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:43.892 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd86effd700 1 -- 172.21.15.132:0/4225528601 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd870107f70 con 0x7fd870106d70 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd86effd700 1 --2- 172.21.15.132:0/4225528601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd868007440 tx=0x7fd868007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd584990e94667e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd86dffb700 1 -- 172.21.15.132:0/4225528601 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd86800b200 con 0x7fd870106d70 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd86dffb700 1 -- 172.21.15.132:0/4225528601 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd86800b3a0 con 0x7fd870106d70 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd86dffb700 1 -- 172.21.15.132:0/4225528601 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd868004030 con 0x7fd870106d70 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd87558f700 1 -- 172.21.15.132:0/4225528601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 msgr2=0x7fd870107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.055+0000 7fd87558f700 1 --2- 172.21.15.132:0/4225528601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd868007440 tx=0x7fd868007750 comp rx=0 tx=0).stop 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 -- 172.21.15.132:0/4225528601 shutdown_connections 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 --2- 172.21.15.132:0/4225528601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870107190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 -- 172.21.15.132:0/4225528601 >> 172.21.15.132:0/4225528601 conn(0x7fd8701023b0 msgr2=0x7fd8701047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:43.893 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 -- 172.21.15.132:0/4225528601 shutdown_connections 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 -- 172.21.15.132:0/4225528601 wait complete. 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 Processor -- start 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.056+0000 7fd87558f700 1 -- start start 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd87558f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd87558f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd87007af80 con 0x7fd870106d70 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd86effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd86effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43712/0 (socket says 172.21.15.132:43712) 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd86effd700 1 -- 172.21.15.132:0/1580799298 learned_addr learned my addr 172.21.15.132:0/1580799298 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:43.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd86effd700 1 -- 172.21.15.132:0/1580799298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd870199ff0 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd86effd700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd8680094f0 tx=0x7fd86800b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8680048a0 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd868003ee0 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd868011e30 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87019a280 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.057+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87019cf70 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.058+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fd868019420 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.058+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8700693f0 con 0x7fd870106d70 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.059+0000 7fd857fff700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 0x7fd85803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:43.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.059+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd868051930 con 0x7fd870106d70 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.059+0000 7fd86e7fc700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 0x7fd85803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.059+0000 7fd86e7fc700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 0x7fd85803b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.064+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd868051690 con 0x7fd870106d70 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.341+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd870193010 con 0x7fd870106d70 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.341+0000 7fd857fff700 1 -- 172.21.15.132:0/1580799298 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd870193010 con 0x7fd870106d70 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 msgr2=0x7fd85803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 0x7fd85803b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 msgr2=0x7fd870199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd8680094f0 tx=0x7fd86800b300 comp rx=0 tx=0).stop 2024-03-18T22:59:43.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 shutdown_connections 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fd858038e80 0x7fd85803b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 --2- 172.21.15.132:0/1580799298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd870106d70 0x7fd870199ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.343+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 >> 172.21.15.132:0/1580799298 conn(0x7fd8701023b0 msgr2=0x7fd8701900f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.344+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 shutdown_connections 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:43.344+0000 7fd87558f700 1 -- 172.21.15.132:0/1580799298 wait complete. 2024-03-18T22:59:43.897 INFO:teuthology.orchestra.run.smithi132.stdout:mgr is available 2024-03-18T22:59:44.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:44 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1562271297' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout fsid = 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T22:59:45.048 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.291+0000 7f9faaee2700 1 Processor -- start 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.291+0000 7f9faaee2700 1 -- start start 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9faaee2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9faaee2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41087b0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.049 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa8c7e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa8c7e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43722/0 (socket says 172.21.15.132:43722) 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa8c7e700 1 -- 172.21.15.132:0/3662084497 learned_addr learned my addr 172.21.15.132:0/3662084497 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa8c7e700 1 -- 172.21.15.132:0/3662084497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa4108fc0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa8c7e700 1 --2- 172.21.15.132:0/3662084497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098a0 tx=0x7f9f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89ba90e4484b85e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa37fe700 1 -- 172.21.15.132:0/3662084497 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f98004160 con 0x7f9fa4107dc0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa37fe700 1 -- 172.21.15.132:0/3662084497 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f98004300 con 0x7f9fa4107dc0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.292+0000 7f9fa37fe700 1 -- 172.21.15.132:0/3662084497 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f98011b50 con 0x7f9fa4107dc0 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 -- 172.21.15.132:0/3662084497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 msgr2=0x7f9fa41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 --2- 172.21.15.132:0/3662084497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098a0 tx=0x7f9f98009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 -- 172.21.15.132:0/3662084497 shutdown_connections 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 --2- 172.21.15.132:0/3662084497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa41081e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:45.050 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 -- 172.21.15.132:0/3662084497 >> 172.21.15.132:0/3662084497 conn(0x7f9fa4103340 msgr2=0x7f9fa41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 -- 172.21.15.132:0/3662084497 shutdown_connections 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.293+0000 7f9faaee2700 1 -- 172.21.15.132:0/3662084497 wait complete. 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9faaee2700 1 Processor -- start 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9faaee2700 1 -- start start 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9faaee2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9faaee2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa410c370 con 0x7f9fa4107dc0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa8c7e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa8c7e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43736/0 (socket says 172.21.15.132:43736) 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa8c7e700 1 -- 172.21.15.132:0/1562271297 learned_addr learned my addr 172.21.15.132:0/1562271297 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa8c7e700 1 -- 172.21.15.132:0/1562271297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa419c2f0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa8c7e700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9f98011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f98003b20 con 0x7f9fa4107dc0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f98003cc0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.051 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa419c580 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.294+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f98024910 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.295+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa419f230 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.295+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f9f98022020 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.296+0000 7f9fa1ffb700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 0x7f9f9403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.296+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa40693f0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.296+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f98011e00 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.296+0000 7f9fa3fff700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 0x7f9f9403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.296+0000 7f9fa3fff700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 0x7f9f9403b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f90006fd0 tx=0x7f9f90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.301+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f9804c8f0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.493+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9fa419f400 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.494+0000 7f9fa1ffb700 1 -- 172.21.15.132:0/1562271297 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7f9f9801acd0 con 0x7f9fa4107dc0 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 msgr2=0x7f9f9403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 0x7f9f9403b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f90006fd0 tx=0x7f9f90006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:45.052 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 msgr2=0x7f9fa419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f98000c00 tx=0x7f9f98011c10 comp rx=0 tx=0).stop 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 shutdown_connections 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9f94038ed0 0x7f9f9403b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 --2- 172.21.15.132:0/1562271297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fa4107dc0 0x7f9fa419bdb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 >> 172.21.15.132:0/1562271297 conn(0x7f9fa4103340 msgr2=0x7f9fa4103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.496+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 shutdown_connections 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:44.497+0000 7f9faaee2700 1 -- 172.21.15.132:0/1562271297 wait complete. 2024-03-18T22:59:45.053 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling cephadm module... 2024-03-18T22:59:46.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:45 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1077256385' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-18T22:59:46.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setuser ceph since I am not root 2024-03-18T22:59:46.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setgroup ceph since I am not root 2024-03-18T22:59:46.394 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f419d214700 1 Processor -- start 2024-03-18T22:59:46.394 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f419d214700 1 -- start start 2024-03-18T22:59:46.394 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f419d214700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:46.394 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f419d214700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f41981076f0 con 0x7f4198106d00 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f4196d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f4196d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43748/0 (socket says 172.21.15.132:43748) 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f4196d9d700 1 -- 172.21.15.132:0/162969771 learned_addr learned my addr 172.21.15.132:0/162969771 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f4196d9d700 1 -- 172.21.15.132:0/162969771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4198107f00 con 0x7f4198106d00 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.485+0000 7f4196d9d700 1 --2- 172.21.15.132:0/162969771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41880098a0 tx=0x7f4188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733c2b7199a161c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f4195d9b700 1 -- 172.21.15.132:0/162969771 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4188004160 con 0x7f4198106d00 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f4195d9b700 1 -- 172.21.15.132:0/162969771 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4188004300 con 0x7f4198106d00 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f4195d9b700 1 -- 172.21.15.132:0/162969771 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4188011b50 con 0x7f4198106d00 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 -- 172.21.15.132:0/162969771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 msgr2=0x7f4198107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 --2- 172.21.15.132:0/162969771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41880098a0 tx=0x7f4188009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 -- 172.21.15.132:0/162969771 shutdown_connections 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 --2- 172.21.15.132:0/162969771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f4198107120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 -- 172.21.15.132:0/162969771 >> 172.21.15.132:0/162969771 conn(0x7f4198102320 msgr2=0x7f4198104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 -- 172.21.15.132:0/162969771 shutdown_connections 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.486+0000 7f419d214700 1 -- 172.21.15.132:0/162969771 wait complete. 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.487+0000 7f419d214700 1 Processor -- start 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.487+0000 7f419d214700 1 -- start start 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.487+0000 7f419d214700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.487+0000 7f419d214700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f419810b2b0 con 0x7f4198106d00 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.487+0000 7f4196d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f4196d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43752/0 (socket says 172.21.15.132:43752) 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f4196d9d700 1 -- 172.21.15.132:0/1077256385 learned_addr learned my addr 172.21.15.132:0/1077256385 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f4196d9d700 1 -- 172.21.15.132:0/1077256385 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419819eaf0 con 0x7f4198106d00 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f4196d9d700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4188011670 tx=0x7f4188011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41880039c0 con 0x7f4198106d00 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419819ecc0 con 0x7f4198106d00 2024-03-18T22:59:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4188011e50 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f418801a5c0 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.488+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419819f0e0 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.489+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f418801a8c0 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.489+0000 7f417ffff700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 0x7f418003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.489+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f418804e0f0 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.489+0000 7f419659c700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 0x7f418003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.489+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198063940 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.490+0000 7f419659c700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 0x7f418003b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.494+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4188019c70 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.741+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f419819f310 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.793+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f419819f310 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.794+0000 7f417ffff700 1 -- 172.21.15.132:0/1077256385 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f418802d910 con 0x7f4198106d00 2024-03-18T22:59:46.397 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 msgr2=0x7f418003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 0x7f418003b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f418c006fd0 tx=0x7f418c006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 msgr2=0x7f419807ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4188011670 tx=0x7f4188011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 shutdown_connections 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f4180038e80 0x7f418003b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 --2- 172.21.15.132:0/1077256385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4198106d00 0x7f419807ba20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 >> 172.21.15.132:0/1077256385 conn(0x7f4198102320 msgr2=0x7f4198102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 shutdown_connections 2024-03-18T22:59:46.398 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:45.796+0000 7f419d214700 1 -- 172.21.15.132:0/1077256385 wait complete. 2024-03-18T22:59:46.455 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:46 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:46.357+0000 7f6be3301200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T22:59:46.455 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:46 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:46.454+0000 7f6be3301200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T22:59:47.058 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:46 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1077256385' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-18T22:59:47.059 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:46 smithi132 ceph-mon[32020]: mgrmap e5: x(active, since 13s) 2024-03-18T22:59:47.351 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.121+0000 7f6be3301200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T22:59:47.352 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.213+0000 7f6be3301200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T22:59:47.618 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.359+0000 7f6be3301200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T22:59:47.618 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.455+0000 7f6be3301200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T22:59:47.618 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.537+0000 7f6be3301200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T22:59:47.661 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:47.661 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-18T22:59:47.661 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.787+0000 7f908f0d8700 1 Processor -- start 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.787+0000 7f908f0d8700 1 -- start start 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.787+0000 7f908f0d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.787+0000 7f908f0d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f908807c8c0 con 0x7f908807de90 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f908ce74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f908ce74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43788/0 (socket says 172.21.15.132:43788) 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f908ce74700 1 -- 172.21.15.132:0/3687993657 learned_addr learned my addr 172.21.15.132:0/3687993657 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f908ce74700 1 -- 172.21.15.132:0/3687993657 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908807ca40 con 0x7f908807de90 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f908ce74700 1 --2- 172.21.15.132:0/3687993657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f907c0098a0 tx=0x7f907c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a036bec253848bc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:47.662 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f90877fe700 1 -- 172.21.15.132:0/3687993657 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c004160 con 0x7f908807de90 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f90877fe700 1 -- 172.21.15.132:0/3687993657 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f907c004300 con 0x7f908807de90 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.788+0000 7f90877fe700 1 -- 172.21.15.132:0/3687993657 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c011b50 con 0x7f908807de90 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 -- 172.21.15.132:0/3687993657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 msgr2=0x7f908807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 --2- 172.21.15.132:0/3687993657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f907c0098a0 tx=0x7f907c009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 -- 172.21.15.132:0/3687993657 shutdown_connections 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 --2- 172.21.15.132:0/3687993657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908807c2f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 -- 172.21.15.132:0/3687993657 >> 172.21.15.132:0/3687993657 conn(0x7f9088103380 msgr2=0x7f90881057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 -- 172.21.15.132:0/3687993657 shutdown_connections 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.789+0000 7f908f0d8700 1 -- 172.21.15.132:0/3687993657 wait complete. 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908f0d8700 1 Processor -- start 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908f0d8700 1 -- start start 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908f0d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:47.663 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908f0d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f908810e3e0 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908ce74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908ce74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43804/0 (socket says 172.21.15.132:43804) 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908ce74700 1 -- 172.21.15.132:0/823474076 learned_addr learned my addr 172.21.15.132:0/823474076 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908ce74700 1 -- 172.21.15.132:0/823474076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908819e4b0 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f908ce74700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f907c011670 tx=0x7f907c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.790+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c0039c0 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.791+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f907c011e50 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.791+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f907c01a5c0 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.791+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908819e740 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.791+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90881a1430 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.791+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f907c024430 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.792+0000 7f9085ffb700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 0x7f907803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.792+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f907c04d1a0 con 0x7f908807de90 2024-03-18T22:59:47.664 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.792+0000 7f9087fff700 1 -- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 msgr2=0x7f907803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.792+0000 7f9087fff700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 0x7f907803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.792+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90880693f0 con 0x7f908807de90 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.797+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f907c017ca0 con 0x7f908807de90 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.993+0000 7f9087fff700 1 -- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 msgr2=0x7f907803b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:46.993+0000 7f9087fff700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 0x7f907803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.058+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f90881a1600 con 0x7f908807de90 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.058+0000 7f9085ffb700 1 -- 172.21.15.132:0/823474076 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f907c003b60 con 0x7f908807de90 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 msgr2=0x7f907803b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 0x7f907803b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.665 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 msgr2=0x7f908819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f907c011670 tx=0x7f907c011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 shutdown_connections 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7f9078038ed0 0x7f907803b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 --2- 172.21.15.132:0/823474076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f908807de90 0x7f908819df70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 >> 172.21.15.132:0/823474076 conn(0x7f9088103380 msgr2=0x7f9088103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 shutdown_connections 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:47.062+0000 7f908f0d8700 1 -- 172.21.15.132:0/823474076 wait complete. 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for the mgr to restart... 2024-03-18T22:59:47.666 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr epoch 5... 2024-03-18T22:59:47.918 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/823474076' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T22:59:47.918 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.806+0000 7f6be3301200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T22:59:47.918 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:47.917+0000 7f6be3301200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T22:59:48.281 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:48.004+0000 7f6be3301200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T22:59:48.281 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:48.104+0000 7f6be3301200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T22:59:48.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:48.280+0000 7f6be3301200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T22:59:49.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:49.244+0000 7f6be3301200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T22:59:49.616 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:49.373+0000 7f6be3301200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T22:59:49.996 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:49.710+0000 7f6be3301200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T22:59:49.997 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:49.800+0000 7f6be3301200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T22:59:49.997 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:49.995+0000 7f6be3301200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:59:50.365 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:50.181+0000 7f6be3301200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T22:59:50.728 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:50.465+0000 7f6be3301200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T22:59:50.728 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:50.727+0000 7f6be3301200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T22:59:51.087 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:50.964+0000 7f6be3301200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T22:59:52.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:51 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:51.761+0000 7f6be3301200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T22:59:52.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:52.348+0000 7f6be3301200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T22:59:52.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:52.446+0000 7f6be3301200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T22:59:53.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T22:59:52.660+0000 7f6be3301200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: Active manager daemon x restarted 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: Activating manager daemon x 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: osdmap e2: 0 total, 0 up, 0 in 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: mgrmap e6: x(active, starting, since 0.00335027s) 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T22:59:53.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: Manager daemon x is now available 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:53.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:52 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: Found migration_current of "None". Setting to last migration. 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: mgrmap e7: x(active, since 1.00554s) 2024-03-18T22:59:53.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:53 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.077+0000 7fb0368fe700 1 Processor -- start 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.077+0000 7fb0368fe700 1 -- start start 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb0368fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb0368fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb03007bd80 con 0x7fb030108950 2024-03-18T22:59:54.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43814/0 (socket says 172.21.15.132:43814) 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb02ffff700 1 -- 172.21.15.132:0/1320940112 learned_addr learned my addr 172.21.15.132:0/1320940112 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb02ffff700 1 -- 172.21.15.132:0/1320940112 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03010b280 con 0x7fb030108950 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.078+0000 7fb02ffff700 1 --2- 172.21.15.132:0/1320940112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098a0 tx=0x7fb020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd0121eb76bcc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb02effd700 1 -- 172.21.15.132:0/1320940112 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb020004160 con 0x7fb030108950 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb02effd700 1 -- 172.21.15.132:0/1320940112 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb020004300 con 0x7fb030108950 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb02effd700 1 -- 172.21.15.132:0/1320940112 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb020011b50 con 0x7fb030108950 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 -- 172.21.15.132:0/1320940112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 msgr2=0x7fb03010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 --2- 172.21.15.132:0/1320940112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098a0 tx=0x7fb020009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 -- 172.21.15.132:0/1320940112 shutdown_connections 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 --2- 172.21.15.132:0/1320940112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb03010ad40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 -- 172.21.15.132:0/1320940112 >> 172.21.15.132:0/1320940112 conn(0x7fb030102320 msgr2=0x7fb030104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 -- 172.21.15.132:0/1320940112 shutdown_connections 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.079+0000 7fb0368fe700 1 -- 172.21.15.132:0/1320940112 wait complete. 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb0368fe700 1 Processor -- start 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb0368fe700 1 -- start start 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb0368fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb0368fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010c1e0 con 0x7fb030108950 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43816/0 (socket says 172.21.15.132:43816) 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.080+0000 7fb02ffff700 1 -- 172.21.15.132:0/3042753769 learned_addr learned my addr 172.21.15.132:0/3042753769 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb02ffff700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb030108710 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb02ffff700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb020011b60 tx=0x7fb020011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb020003b20 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb020003cc0 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb030106900 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb020024840 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.081+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb030106e40 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fb020022020 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02d7fa700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb01803baf0 con 0x7fb018038f20 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb020011e00 con 0x7fb030108950 2024-03-18T22:59:54.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02f7fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.082+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.283+0000 7fb02f7fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.283+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.684+0000 7fb02f7fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:48.684+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:49.485+0000 7fb02f7fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:49.485+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:51.086+0000 7fb02f7fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/1223581697 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:51.086+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:52.675+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7fb020024b10 con 0x7fb030108950 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:52.675+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 msgr2=0x7fb01803b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:52.675+0000 7fb02d7fa700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.676+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7fb02004ee80 con 0x7fb030108950 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.676+0000 7fb02d7fa700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 0x7fb01803ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.677+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb0200470e0 con 0x7fb01803c810 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.677+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 0x7fb01803ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.677+0000 7fb02f7fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 0x7fb01803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb024003a10 tx=0x7fb024009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.678+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb0200470e0 con 0x7fb01803c810 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.684+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb0300693f0 con 0x7fb01803c810 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.684+0000 7fb02d7fa700 1 -- 172.21.15.132:0/3042753769 <== mgr.14120 v2:172.21.15.132:6800/3371561746 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb0300693f0 con 0x7fb01803c810 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 msgr2=0x7fb01803ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 0x7fb01803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb024003a10 tx=0x7fb024009290 comp rx=0 tx=0).stop 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 msgr2=0x7fb0301081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb020011b60 tx=0x7fb020011c40 comp rx=0 tx=0).stop 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 shutdown_connections 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb01803c810 0x7fb01803ec00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb030108950 0x7fb0301081d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 --2- 172.21.15.132:0/3042753769 >> [v2:172.21.15.132:6800/1223581697,v1:172.21.15.132:6801/1223581697] conn(0x7fb018038f20 0x7fb01803b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 >> 172.21.15.132:0/3042753769 conn(0x7fb030102320 msgr2=0x7fb030102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 shutdown_connections 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:53.685+0000 7fb0368fe700 1 -- 172.21.15.132:0/3042753769 wait complete. 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:mgr epoch 5 is available 2024-03-18T22:59:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:Setting orchestrator backend to cephadm... 2024-03-18T22:59:54.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: [18/Mar/2024:22:59:53] ENGINE Bus STARTING 2024-03-18T22:59:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: [18/Mar/2024:22:59:53] ENGINE Serving on http://172.21.15.132:8765 2024-03-18T22:59:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: [18/Mar/2024:22:59:53] ENGINE Serving on https://172.21.15.132:7150 2024-03-18T22:59:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: [18/Mar/2024:22:59:53] ENGINE Bus STARTED 2024-03-18T22:59:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T22:59:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:54 smithi132 ceph-mon[32020]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f04116c6700 1 Processor -- start 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f04116c6700 1 -- start start 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f04116c6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f04116c6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f040c07be60 con 0x7f040c108820 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f040affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f040affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41134/0 (socket says 172.21.15.132:41134) 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f040affd700 1 -- 172.21.15.132:0/919999463 learned_addr learned my addr 172.21.15.132:0/919999463 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:55.474 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.688+0000 7f040affd700 1 -- 172.21.15.132:0/919999463 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c10b150 con 0x7f040c108820 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f040affd700 1 --2- 172.21.15.132:0/919999463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098a0 tx=0x7f03fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d244a545ebf73c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f0409ffb700 1 -- 172.21.15.132:0/919999463 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc004160 con 0x7f040c108820 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f0409ffb700 1 -- 172.21.15.132:0/919999463 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03fc004300 con 0x7f040c108820 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f0409ffb700 1 -- 172.21.15.132:0/919999463 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc011b50 con 0x7f040c108820 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 -- 172.21.15.132:0/919999463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 msgr2=0x7f040c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 --2- 172.21.15.132:0/919999463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0098a0 tx=0x7f03fc009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 -- 172.21.15.132:0/919999463 shutdown_connections 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 --2- 172.21.15.132:0/919999463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c10ac10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 -- 172.21.15.132:0/919999463 >> 172.21.15.132:0/919999463 conn(0x7f040c1022d0 msgr2=0x7f040c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 -- 172.21.15.132:0/919999463 shutdown_connections 2024-03-18T22:59:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.689+0000 7f04116c6700 1 -- 172.21.15.132:0/919999463 wait complete. 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f04116c6700 1 Processor -- start 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f04116c6700 1 -- start start 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f04116c6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f04116c6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f040c10c180 con 0x7f040c108820 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f040affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f040affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41150/0 (socket says 172.21.15.132:41150) 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.690+0000 7f040affd700 1 -- 172.21.15.132:0/270025924 learned_addr learned my addr 172.21.15.132:0/270025924 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f040affd700 1 -- 172.21.15.132:0/270025924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c19c2f0 con 0x7f040c108820 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f040affd700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f03fc011670 tx=0x7f03fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc0039c0 con 0x7f040c108820 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03fc011e50 con 0x7f040c108820 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c19c580 con 0x7f040c108820 2024-03-18T22:59:55.476 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03fc01a5c0 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.691+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c19f270 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f03fc01a870 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f03f3fff700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 0x7f03f403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f03fc024a00 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f040a7fc700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 0x7f03f403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c063940 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.692+0000 7f040a7fc700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 0x7f03f403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.697+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03fc019c70 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.727+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f03fc02d910 con 0x7f040c108820 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.911+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f040c103520 con 0x7f03f4038e10 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.917+0000 7f03f3fff700 1 -- 172.21.15.132:0/270025924 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f040c103520 con 0x7f03f4038e10 2024-03-18T22:59:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 msgr2=0x7f03f403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 0x7f03f403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0400006fd0 tx=0x7f0400006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 msgr2=0x7f040c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f03fc011670 tx=0x7f03fc011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 shutdown_connections 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f03f4038e10 0x7f03f403b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 --2- 172.21.15.132:0/270025924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f040c108820 0x7f040c19bdb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 >> 172.21.15.132:0/270025924 conn(0x7f040c1022d0 msgr2=0x7f040c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 shutdown_connections 2024-03-18T22:59:55.478 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:54.920+0000 7f04116c6700 1 -- 172.21.15.132:0/270025924 wait complete. 2024-03-18T22:59:56.082 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:55 smithi132 ceph-mon[32020]: mgrmap e8: x(active, since 2s) 2024-03-18T22:59:56.082 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:55 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:56.082 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:55 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb768166700 1 Processor -- start 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb768166700 1 -- start start 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb768166700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb768166700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb76007be20 con 0x7fb7601088b0 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb765f02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:56.588 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb765f02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41160/0 (socket says 172.21.15.132:41160) 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.875+0000 7fb765f02700 1 -- 172.21.15.132:0/4232475998 learned_addr learned my addr 172.21.15.132:0/4232475998 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb765f02700 1 -- 172.21.15.132:0/4232475998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76010b1e0 con 0x7fb7601088b0 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb765f02700 1 --2- 172.21.15.132:0/4232475998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cf6d3cfedf8672 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb764f00700 1 -- 172.21.15.132:0/4232475998 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c004160 con 0x7fb7601088b0 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb764f00700 1 -- 172.21.15.132:0/4232475998 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c004300 con 0x7fb7601088b0 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb764f00700 1 -- 172.21.15.132:0/4232475998 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c011b50 con 0x7fb7601088b0 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb768166700 1 -- 172.21.15.132:0/4232475998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 msgr2=0x7fb76010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.876+0000 7fb768166700 1 --2- 172.21.15.132:0/4232475998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- 172.21.15.132:0/4232475998 shutdown_connections 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 --2- 172.21.15.132:0/4232475998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb76010aca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:56.589 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- 172.21.15.132:0/4232475998 >> 172.21.15.132:0/4232475998 conn(0x7fb760102320 msgr2=0x7fb760104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- 172.21.15.132:0/4232475998 shutdown_connections 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- 172.21.15.132:0/4232475998 wait complete. 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 Processor -- start 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- start start 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.877+0000 7fb768166700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb76010c200 con 0x7fb7601088b0 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb765f02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb765f02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41164/0 (socket says 172.21.15.132:41164) 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb765f02700 1 -- 172.21.15.132:0/2726170679 learned_addr learned my addr 172.21.15.132:0/2726170679 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb765f02700 1 -- 172.21.15.132:0/2726170679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760108670 con 0x7fb7601088b0 2024-03-18T22:59:56.590 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb765f02700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011670 tx=0x7fb75c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c0039c0 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c011e50 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb760106860 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c01a5c0 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.878+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb760106da0 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.879+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fb75c01a8c0 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.879+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7600693f0 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.879+0000 7fb756ffd700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 0x7fb74c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.879+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb75c04d250 con 0x7fb7601088b0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.879+0000 7fb765701700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 0x7fb74c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:56.591 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.880+0000 7fb765701700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 0x7fb74c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:55.884+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb75c016260 con 0x7fb7601088b0 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.081+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb760103570 con 0x7fb74c038ed0 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.082+0000 7fb756ffd700 1 -- 172.21.15.132:0/2726170679 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb760103570 con 0x7fb74c038ed0 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 msgr2=0x7fb74c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 0x7fb74c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 msgr2=0x7fb760108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011670 tx=0x7fb75c011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 shutdown_connections 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fb74c038ed0 0x7fb74c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 --2- 172.21.15.132:0/2726170679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7601088b0 0x7fb760108130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:56.592 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 >> 172.21.15.132:0/2726170679 conn(0x7fb760102320 msgr2=0x7fb760102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:56.593 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 shutdown_connections 2024-03-18T22:59:56.593 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.084+0000 7fb768166700 1 -- 172.21.15.132:0/2726170679 wait complete. 2024-03-18T22:59:56.593 INFO:teuthology.orchestra.run.smithi132.stdout:Generating ssh key... 2024-03-18T22:59:56.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:56 smithi132 ceph-mon[32020]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-mon[32020]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: Generating public/private rsa key pair. 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: Your identification has been saved in /tmp/tmpzdcarjyq/key. 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: Your public key has been saved in /tmp/tmpzdcarjyq/key.pub. 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: The key fingerprint is: 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: SHA256:9ySaF+M89H/we4WRcN8WYHOG8fsOKxIYSUNaDNSujd8 ceph-0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:58.044 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: The key's randomart image is: 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: +---[RSA 3072]----+ 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | .o=o =+o | 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | o= o.*. | 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | .o o o +o| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | + o =| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | +So* . = | 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | o oB.B o o| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | .o.=.o .oo| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | ..E... ++| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: | . .oo+| 2024-03-18T22:59:58.045 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 22:59:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: +----[SHA256]-----+ 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc7892ea700 1 Processor -- start 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc7892ea700 1 -- start start 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc7892ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc7892ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc78407bdc0 con 0x7fc784108900 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41170/0 (socket says 172.21.15.132:41170) 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.978+0000 7fc782ffd700 1 -- 172.21.15.132:0/688935959 learned_addr learned my addr 172.21.15.132:0/688935959 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.979+0000 7fc782ffd700 1 -- 172.21.15.132:0/688935959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78410b230 con 0x7fc784108900 2024-03-18T22:59:58.287 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.979+0000 7fc782ffd700 1 --2- 172.21.15.132:0/688935959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009e80 tx=0x7fc77c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9a88deecb92249 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.979+0000 7fc781ffb700 1 -- 172.21.15.132:0/688935959 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc77c017070 con 0x7fc784108900 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.979+0000 7fc781ffb700 1 -- 172.21.15.132:0/688935959 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc77c00b8c0 con 0x7fc784108900 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.979+0000 7fc781ffb700 1 -- 172.21.15.132:0/688935959 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc77c0045c0 con 0x7fc784108900 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- 172.21.15.132:0/688935959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 msgr2=0x7fc78410acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 --2- 172.21.15.132:0/688935959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009e80 tx=0x7fc77c00b330 comp rx=0 tx=0).stop 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- 172.21.15.132:0/688935959 shutdown_connections 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 --2- 172.21.15.132:0/688935959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78410acf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- 172.21.15.132:0/688935959 >> 172.21.15.132:0/688935959 conn(0x7fc784102290 msgr2=0x7fc7841046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- 172.21.15.132:0/688935959 shutdown_connections 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- 172.21.15.132:0/688935959 wait complete. 2024-03-18T22:59:58.288 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 Processor -- start 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.980+0000 7fc7892ea700 1 -- start start 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc7892ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc7892ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410c1f0 con 0x7fc784108900 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41180/0 (socket says 172.21.15.132:41180) 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc782ffd700 1 -- 172.21.15.132:0/3636687733 learned_addr learned my addr 172.21.15.132:0/3636687733 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc782ffd700 1 -- 172.21.15.132:0/3636687733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78419c210 con 0x7fc784108900 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc782ffd700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0072a0 tx=0x7fc77c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc77c020020 con 0x7fc784108900 2024-03-18T22:59:58.289 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc77c003b50 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78419c4a0 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc77c017040 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.981+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78419f180 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.982+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fc77c023020 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.982+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7840693f0 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.982+0000 7fc76bfff700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 0x7fc76c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.982+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc77c00f050 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.982+0000 7fc7827fc700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 0x7fc76c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.983+0000 7fc7827fc700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 0x7fc76c03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:56.987+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc77c02a020 con 0x7fc784108900 2024-03-18T22:59:58.290 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.184+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fc784103450 con 0x7fc76c038f20 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.685+0000 7fc76bfff700 1 -- 172.21.15.132:0/3636687733 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc784103450 con 0x7fc76c038f20 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 msgr2=0x7fc76c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 0x7fc76c03b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 msgr2=0x7fc78419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0072a0 tx=0x7fc77c00bd50 comp rx=0 tx=0).stop 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 shutdown_connections 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fc76c038f20 0x7fc76c03b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 --2- 172.21.15.132:0/3636687733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc784108900 0x7fc78419bcd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:58.291 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.687+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 >> 172.21.15.132:0/3636687733 conn(0x7fc784102290 msgr2=0x7fc784102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:58.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.688+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 shutdown_connections 2024-03-18T22:59:58.292 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:57.688+0000 7fc7892ea700 1 -- 172.21.15.132:0/3636687733 wait complete. 2024-03-18T22:59:58.905 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:58 smithi132 ceph-mon[32020]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T22:59:58.905 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:58 smithi132 ceph-mon[32020]: Generating ssh key... 2024-03-18T22:59:59.449 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.698+0000 7f7f4e9d2700 1 Processor -- start 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.698+0000 7f7f4e9d2700 1 -- start start 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.698+0000 7f7f4e9d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.698+0000 7f7f4e9d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4807be00 con 0x7f7f481066c0 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.698+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41184/0 (socket says 172.21.15.132:41184) 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f47fff700 1 -- 172.21.15.132:0/3196116645 learned_addr learned my addr 172.21.15.132:0/3196116645 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f47fff700 1 -- 172.21.15.132:0/3196116645 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48107020 con 0x7f7f481066c0 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f47fff700 1 --2- 172.21.15.132:0/3196116645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf6ef29f1c7125 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f46ffd700 1 -- 172.21.15.132:0/3196116645 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38004160 con 0x7f7f481066c0 2024-03-18T22:59:59.450 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f46ffd700 1 -- 172.21.15.132:0/3196116645 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f38004300 con 0x7f7f481066c0 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.699+0000 7f7f46ffd700 1 -- 172.21.15.132:0/3196116645 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38011b50 con 0x7f7f481066c0 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/3196116645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 msgr2=0x7f7f48106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/3196116645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).stop 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/3196116645 shutdown_connections 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/3196116645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f48106ae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/3196116645 >> 172.21.15.132:0/3196116645 conn(0x7f7f48102320 msgr2=0x7f7f48104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/3196116645 shutdown_connections 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/3196116645 wait complete. 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 Processor -- start 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.700+0000 7f7f4e9d2700 1 -- start start 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f4e9d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f4e9d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481080b0 con 0x7f7f481066c0 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:59.451 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41200/0 (socket says 172.21.15.132:41200) 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f47fff700 1 -- 172.21.15.132:0/2161388811 learned_addr learned my addr 172.21.15.132:0/2161388811 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f47fff700 1 -- 172.21.15.132:0/2161388811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819e4c0 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f47fff700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7f38011670 tx=0x7f7f38011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f380039c0 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f38011e50 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3801a5c0 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4819e750 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.701+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a1440 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.702+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f7f38024430 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.702+0000 7f7f457fa700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 0x7f7f3003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.702+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f480693f0 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.702+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f3804d290 con 0x7f7f481066c0 2024-03-18T22:59:59.452 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.703+0000 7f7f477fe700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 0x7f7f3003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.703+0000 7f7f477fe700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 0x7f7f3003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006fd0 tx=0x7f7f3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.708+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f3802d910 con 0x7f7f481066c0 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.904+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7f4802de00 con 0x7f7f30038f20 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.905+0000 7f7f457fa700 1 -- 172.21.15.132:0/2161388811 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7f4802de00 con 0x7f7f30038f20 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 msgr2=0x7f7f3003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 0x7f7f3003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006fd0 tx=0x7f7f3c006e40 comp rx=0 tx=0).stop 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 msgr2=0x7f7f4819df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7f38011670 tx=0x7f7f38011b60 comp rx=0 tx=0).stop 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 shutdown_connections 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f7f30038f20 0x7f7f3003b3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 --2- 172.21.15.132:0/2161388811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f481066c0 0x7f7f4819df80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 >> 172.21.15.132:0/2161388811 conn(0x7f7f48102320 msgr2=0x7f7f48194c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 shutdown_connections 2024-03-18T22:59:59.453 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:58.907+0000 7f7f4e9d2700 1 -- 172.21.15.132:0/2161388811 wait complete. 2024-03-18T22:59:59.454 INFO:teuthology.orchestra.run.smithi132.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-18T22:59:59.454 INFO:teuthology.orchestra.run.smithi132.stdout:Adding key to root@localhost authorized_keys... 2024-03-18T22:59:59.454 INFO:teuthology.orchestra.run.smithi132.stdout:Adding host smithi132... 2024-03-18T23:00:00.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 22:59:59 smithi132 ceph-mon[32020]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:01.023 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:00 smithi132 ceph-mon[32020]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "addr": "172.21.15.132", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:01.024 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:00 smithi132 ceph-mon[32020]: Deploying cephadm binary to smithi132 2024-03-18T23:00:04.835 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Added host 'smithi132' with addr '172.21.15.132' 2024-03-18T23:00:04.837 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff8145db700 1 Processor -- start 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff8145db700 1 -- start start 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff8145db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff8145db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1087f0 con 0x7ff80c107e90 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff812377700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff812377700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41206/0 (socket says 172.21.15.132:41206) 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff812377700 1 -- 172.21.15.132:0/3390930205 learned_addr learned my addr 172.21.15.132:0/3390930205 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.839+0000 7ff812377700 1 -- 172.21.15.132:0/3390930205 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c108fa0 con 0x7ff80c107e90 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff812377700 1 --2- 172.21.15.132:0/3390930205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1e45078a13271c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:04.838 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff811375700 1 -- 172.21.15.132:0/3390930205 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff808004160 con 0x7ff80c107e90 2024-03-18T23:00:04.843 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff811375700 1 -- 172.21.15.132:0/3390930205 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff808004300 con 0x7ff80c107e90 2024-03-18T23:00:04.843 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff811375700 1 -- 172.21.15.132:0/3390930205 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff808011b50 con 0x7ff80c107e90 2024-03-18T23:00:04.843 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff8145db700 1 -- 172.21.15.132:0/3390930205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 msgr2=0x7ff80c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.840+0000 7ff8145db700 1 --2- 172.21.15.132:0/3390930205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- 172.21.15.132:0/3390930205 shutdown_connections 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 --2- 172.21.15.132:0/3390930205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c1082b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- 172.21.15.132:0/3390930205 >> 172.21.15.132:0/3390930205 conn(0x7ff80c103470 msgr2=0x7ff80c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- 172.21.15.132:0/3390930205 shutdown_connections 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- 172.21.15.132:0/3390930205 wait complete. 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 Processor -- start 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- start start 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff8145db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c10c260 con 0x7ff80c107e90 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff812377700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff812377700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41214/0 (socket says 172.21.15.132:41214) 2024-03-18T23:00:04.844 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.841+0000 7ff812377700 1 -- 172.21.15.132:0/3866883215 learned_addr learned my addr 172.21.15.132:0/3866883215 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff812377700 1 -- 172.21.15.132:0/3866883215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c19c280 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff812377700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff808011670 tx=0x7ff808011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8080039c0 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff808011e50 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c19c510 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff80801a5c0 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.842+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c19f200 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.843+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7ff80801a8c0 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.843+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c10c3e0 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.843+0000 7ff8037fe700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 0x7ff7f803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.843+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff80804d250 con 0x7ff80c107e90 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.843+0000 7ff811b76700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 0x7ff7f803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.844+0000 7ff811b76700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 0x7ff7f803b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:04.845 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T22:59:59.848+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff808016260 con 0x7ff80c107e90 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:00.048+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "addr": "172.21.15.132", "target": ["mon-mgr", ""]}) -- 0x7ff80c104640 con 0x7ff7f8038ed0 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.230+0000 7ff8037fe700 1 -- 172.21.15.132:0/3866883215 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff80c104640 con 0x7ff7f8038ed0 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 msgr2=0x7ff7f803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 0x7ff7f803b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 msgr2=0x7ff80c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff808011670 tx=0x7ff808011b60 comp rx=0 tx=0).stop 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 shutdown_connections 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7ff7f8038ed0 0x7ff7f803b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:04.846 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 --2- 172.21.15.132:0/3866883215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff80c107e90 0x7ff80c19bd40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:04.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.232+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 >> 172.21.15.132:0/3866883215 conn(0x7ff80c103470 msgr2=0x7ff80c103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:04.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.233+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 shutdown_connections 2024-03-18T23:00:04.847 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:04.233+0000 7ff8145db700 1 -- 172.21.15.132:0/3866883215 wait complete. 2024-03-18T23:00:04.847 INFO:teuthology.orchestra.run.smithi132.stdout:Deploying unmanaged mon service... 2024-03-18T23:00:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:05 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:05 smithi132 ceph-mon[32020]: Added host smithi132 2024-03-18T23:00:05.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:05 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:06.175 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.236+0000 7f01c09e1700 1 Processor -- start 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.236+0000 7f01c09e1700 1 -- start start 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01c09e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01c09e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc108cb0 con 0x7f01bc1082c0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48994/0 (socket says 172.21.15.132:48994) 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01ba59c700 1 -- 172.21.15.132:0/1233400171 learned_addr learned my addr 172.21.15.132:0/1233400171 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01ba59c700 1 -- 172.21.15.132:0/1233400171 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc106f20 con 0x7f01bc1082c0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01ba59c700 1 --2- 172.21.15.132:0/1233400171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e80 tx=0x7f01b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52771daaca9e6f9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01b959a700 1 -- 172.21.15.132:0/1233400171 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b4017070 con 0x7f01bc1082c0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01b959a700 1 -- 172.21.15.132:0/1233400171 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01b400b8c0 con 0x7f01bc1082c0 2024-03-18T23:00:06.176 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.237+0000 7f01b959a700 1 -- 172.21.15.132:0/1233400171 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b40045c0 con 0x7f01bc1082c0 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 -- 172.21.15.132:0/1233400171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 msgr2=0x7f01bc1086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 --2- 172.21.15.132:0/1233400171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e80 tx=0x7f01b400b330 comp rx=0 tx=0).stop 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 -- 172.21.15.132:0/1233400171 shutdown_connections 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 --2- 172.21.15.132:0/1233400171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1086e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 -- 172.21.15.132:0/1233400171 >> 172.21.15.132:0/1233400171 conn(0x7f01bc101a50 msgr2=0x7f01bc103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 -- 172.21.15.132:0/1233400171 shutdown_connections 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.238+0000 7f01c09e1700 1 -- 172.21.15.132:0/1233400171 wait complete. 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01c09e1700 1 Processor -- start 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01c09e1700 1 -- start start 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01c09e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01c09e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10b2e0 con 0x7f01bc1082c0 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:06.177 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49006/0 (socket says 172.21.15.132:49006) 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01ba59c700 1 -- 172.21.15.132:0/3005821569 learned_addr learned my addr 172.21.15.132:0/3005821569 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01ba59c700 1 -- 172.21.15.132:0/3005821569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc19ea90 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.239+0000 7f01ba59c700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e50 tx=0x7f01b400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b4020020 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01b4003b50 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc19ec60 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b4017040 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc19f080 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.240+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f01b4023020 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.241+0000 7f01ab7fe700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 0x7f01a403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.241+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f01b400f050 con 0x7f01bc1082c0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.241+0000 7f01b9d9b700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 0x7f01a403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:06.178 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.241+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc063940 con 0x7f01bc1082c0 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.241+0000 7f01b9d9b700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 0x7f01a403b390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01b0006fd0 tx=0x7f01b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.246+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f01b402a020 con 0x7f01bc1082c0 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.476+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f01bc102c40 con 0x7f01a4038ed0 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.482+0000 7f01ab7fe700 1 -- 172.21.15.132:0/3005821569 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f01bc102c40 con 0x7f01a4038ed0 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 msgr2=0x7f01a403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 0x7f01a403b390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01b0006fd0 tx=0x7f01b0006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 msgr2=0x7f01bc1068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009e50 tx=0x7f01b400bd30 comp rx=0 tx=0).stop 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 shutdown_connections 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f01a4038ed0 0x7f01a403b390 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 --2- 172.21.15.132:0/3005821569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01bc1082c0 0x7f01bc1068b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 >> 172.21.15.132:0/3005821569 conn(0x7f01bc101a50 msgr2=0x7f01bc102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 shutdown_connections 2024-03-18T23:00:06.180 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:05.484+0000 7f01c09e1700 1 -- 172.21.15.132:0/3005821569 wait complete. 2024-03-18T23:00:06.180 INFO:teuthology.orchestra.run.smithi132.stdout:Deploying unmanaged mgr service... 2024-03-18T23:00:06.686 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:06 smithi132 ceph-mon[32020]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:06.686 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:06 smithi132 ceph-mon[32020]: Saving service mon spec with placement count:5 2024-03-18T23:00:06.686 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:06 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:07.575 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:07 smithi132 ceph-mon[32020]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:07.575 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:07 smithi132 ceph-mon[32020]: Saving service mgr spec with placement count:2 2024-03-18T23:00:07.576 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:07 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:08.204 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-18T23:00:08.204 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea32beb700 1 Processor -- start 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea32beb700 1 -- start start 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea32beb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea32beb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c07be40 con 0x7fea2c1088b0 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea30987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea30987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49012/0 (socket says 172.21.15.132:49012) 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea30987700 1 -- 172.21.15.132:0/3410135530 learned_addr learned my addr 172.21.15.132:0/3410135530 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.619+0000 7fea30987700 1 -- 172.21.15.132:0/3410135530 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c10b1e0 con 0x7fea2c1088b0 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea30987700 1 --2- 172.21.15.132:0/3410135530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1579797b5e47a7e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:08.205 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea2b7fe700 1 -- 172.21.15.132:0/3410135530 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea20004160 con 0x7fea2c1088b0 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea2b7fe700 1 -- 172.21.15.132:0/3410135530 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea20004300 con 0x7fea2c1088b0 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea2b7fe700 1 -- 172.21.15.132:0/3410135530 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea20011b50 con 0x7fea2c1088b0 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 -- 172.21.15.132:0/3410135530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 msgr2=0x7fea2c10aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 --2- 172.21.15.132:0/3410135530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea200098a0 tx=0x7fea20009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 -- 172.21.15.132:0/3410135530 shutdown_connections 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 --2- 172.21.15.132:0/3410135530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c10aca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 -- 172.21.15.132:0/3410135530 >> 172.21.15.132:0/3410135530 conn(0x7fea2c1022e0 msgr2=0x7fea2c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 -- 172.21.15.132:0/3410135530 shutdown_connections 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.620+0000 7fea32beb700 1 -- 172.21.15.132:0/3410135530 wait complete. 2024-03-18T23:00:08.206 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea32beb700 1 Processor -- start 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea32beb700 1 -- start start 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea32beb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea32beb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c10c190 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea30987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea30987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49024/0 (socket says 172.21.15.132:49024) 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea30987700 1 -- 172.21.15.132:0/1256585330 learned_addr learned my addr 172.21.15.132:0/1256585330 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea30987700 1 -- 172.21.15.132:0/1256585330 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19c250 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.621+0000 7fea30987700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea20011670 tx=0x7fea20011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.622+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea200039c0 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.622+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea20011e50 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.622+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea2001a5c0 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.622+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19c540 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.622+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c19f1c0 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fea20024430 con 0x7fea2c1088b0 2024-03-18T23:00:08.207 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c063940 con 0x7fea2c1088b0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea29ffb700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 0x7fea1c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea2004d230 con 0x7fea2c1088b0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea2bfff700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 0x7fea1c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.623+0000 7fea2bfff700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 0x7fea1c03b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea18006fd0 tx=0x7fea18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.628+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea2002d490 con 0x7fea2c1088b0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.845+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fea2c19f390 con 0x7fea1c038ed0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.851+0000 7fea29ffb700 1 -- 172.21.15.132:0/1256585330 <== mgr.14120 v2:172.21.15.132:6800/3371561746 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fea2c19f390 con 0x7fea1c038ed0 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 msgr2=0x7fea1c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 0x7fea1c03b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea18006fd0 tx=0x7fea18006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 msgr2=0x7fea2c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea20011670 tx=0x7fea20011b60 comp rx=0 tx=0).stop 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 shutdown_connections 2024-03-18T23:00:08.208 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7fea1c038ed0 0x7fea1c03b390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:08.209 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 --2- 172.21.15.132:0/1256585330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2c1088b0 0x7fea2c19bd10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:08.209 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 >> 172.21.15.132:0/1256585330 conn(0x7fea2c1022e0 msgr2=0x7fea2c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:08.209 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 shutdown_connections 2024-03-18T23:00:08.209 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:06.854+0000 7fea32beb700 1 -- 172.21.15.132:0/1256585330 wait complete. 2024-03-18T23:00:08.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:08 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:08.837 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:08 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:09.561 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.606+0000 7efc06062700 1 Processor -- start 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.606+0000 7efc06062700 1 -- start start 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.606+0000 7efc06062700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.606+0000 7efc06062700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efc0007be20 con 0x7efc001088b0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49026/0 (socket says 172.21.15.132:49026) 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbff7fe700 1 -- 172.21.15.132:0/2487111081 learned_addr learned my addr 172.21.15.132:0/2487111081 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbff7fe700 1 -- 172.21.15.132:0/2487111081 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0010b1e0 con 0x7efc001088b0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbff7fe700 1 --2- 172.21.15.132:0/2487111081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efbf00098a0 tx=0x7efbf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2758fbc5f1c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.607+0000 7efbfe7fc700 1 -- 172.21.15.132:0/2487111081 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0004160 con 0x7efc001088b0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efbfe7fc700 1 -- 172.21.15.132:0/2487111081 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf0004300 con 0x7efc001088b0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efbfe7fc700 1 -- 172.21.15.132:0/2487111081 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0011b20 con 0x7efc001088b0 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 -- 172.21.15.132:0/2487111081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 msgr2=0x7efc0010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.562 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 --2- 172.21.15.132:0/2487111081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efbf00098a0 tx=0x7efbf0009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 -- 172.21.15.132:0/2487111081 shutdown_connections 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 --2- 172.21.15.132:0/2487111081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0010aca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 -- 172.21.15.132:0/2487111081 >> 172.21.15.132:0/2487111081 conn(0x7efc001022e0 msgr2=0x7efc00104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 -- 172.21.15.132:0/2487111081 shutdown_connections 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.608+0000 7efc06062700 1 -- 172.21.15.132:0/2487111081 wait complete. 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.609+0000 7efc06062700 1 Processor -- start 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.609+0000 7efc06062700 1 -- start start 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.609+0000 7efc06062700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.609+0000 7efc06062700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efc0010c240 con 0x7efc001088b0 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49028/0 (socket says 172.21.15.132:49028) 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbff7fe700 1 -- 172.21.15.132:0/4039671016 learned_addr learned my addr 172.21.15.132:0/4039671016 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbff7fe700 1 -- 172.21.15.132:0/4039671016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0019c240 con 0x7efc001088b0 2024-03-18T23:00:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbff7fe700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efbf00117e0 tx=0x7efbf0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0003bf0 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf001a910 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0019c4d0 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efbf0024970 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.610+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0019c930 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.611+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc000693f0 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.612+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7efbf0022020 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.612+0000 7efbfcff9700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 0x7efbe803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.612+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efbf004d150 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.612+0000 7efbfeffd700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 0x7efbe803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.613+0000 7efbfeffd700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 0x7efbe803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006fd0 tx=0x7efbf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.620+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efbf0011e20 con 0x7efc001088b0 2024-03-18T23:00:09.564 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.836+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7efc0019f6a0 con 0x7efc001088b0 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.841+0000 7efbfcff9700 1 -- 172.21.15.132:0/4039671016 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7efbf002ddb0 con 0x7efc001088b0 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 msgr2=0x7efbe803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 0x7efbe803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006fd0 tx=0x7efbf4006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 msgr2=0x7efc0019bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efbf00117e0 tx=0x7efbf0011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 shutdown_connections 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7efbe8038f20 0x7efbe803b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 --2- 172.21.15.132:0/4039671016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc001088b0 0x7efc0019bd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 >> 172.21.15.132:0/4039671016 conn(0x7efc001022e0 msgr2=0x7efc00102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 shutdown_connections 2024-03-18T23:00:09.565 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:08.843+0000 7efc06062700 1 -- 172.21.15.132:0/4039671016 wait complete. 2024-03-18T23:00:10.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:09 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4039671016' entity='client.admin' 2024-03-18T23:00:11.094 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.286+0000 7f3939b09700 1 Processor -- start 2024-03-18T23:00:11.094 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.286+0000 7f3939b09700 1 -- start start 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.286+0000 7f3939b09700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.286+0000 7f3939b09700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f393407be20 con 0x7f3934108860 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49042/0 (socket says 172.21.15.132:49042) 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39337fe700 1 -- 172.21.15.132:0/3997630405 learned_addr learned my addr 172.21.15.132:0/3997630405 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39337fe700 1 -- 172.21.15.132:0/3997630405 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393410b190 con 0x7f3934108860 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39337fe700 1 --2- 172.21.15.132:0/3997630405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39240098a0 tx=0x7f3924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bfc4d9e12b2e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39327fc700 1 -- 172.21.15.132:0/3997630405 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3924004160 con 0x7f3934108860 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39327fc700 1 -- 172.21.15.132:0/3997630405 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3924004300 con 0x7f3934108860 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.287+0000 7f39327fc700 1 -- 172.21.15.132:0/3997630405 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3924011b20 con 0x7f3934108860 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 -- 172.21.15.132:0/3997630405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 msgr2=0x7f393410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 --2- 172.21.15.132:0/3997630405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39240098a0 tx=0x7f3924009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:11.095 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 -- 172.21.15.132:0/3997630405 shutdown_connections 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 --2- 172.21.15.132:0/3997630405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393410ac50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 -- 172.21.15.132:0/3997630405 >> 172.21.15.132:0/3997630405 conn(0x7f39341022d0 msgr2=0x7f39341046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 -- 172.21.15.132:0/3997630405 shutdown_connections 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.288+0000 7f3939b09700 1 -- 172.21.15.132:0/3997630405 wait complete. 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f3939b09700 1 Processor -- start 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f3939b09700 1 -- start start 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f3939b09700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f3939b09700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f393410c1f0 con 0x7f3934108860 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f39337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f39337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49050/0 (socket says 172.21.15.132:49050) 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.289+0000 7f39337fe700 1 -- 172.21.15.132:0/2355791753 learned_addr learned my addr 172.21.15.132:0/2355791753 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f39337fe700 1 -- 172.21.15.132:0/2355791753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393419c250 con 0x7f3934108860 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f39337fe700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39240117e0 tx=0x7f3924011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:11.096 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3924003bf0 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393419c4e0 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392401a910 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3924024970 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.290+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393419c940 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.291+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f3924022020 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.291+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3934063940 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.291+0000 7f3930ff9700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 0x7f391c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.291+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f392404d140 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.291+0000 7f3932ffd700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 0x7f391c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.292+0000 7f3932ffd700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 0x7f391c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.296+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3924011e20 con 0x7f3934108860 2024-03-18T23:00:11.097 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.502+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f393419f3a0 con 0x7f3934108860 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.504+0000 7f3930ff9700 1 -- 172.21.15.132:0/2355791753 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f392400ba00 con 0x7f3934108860 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.507+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 msgr2=0x7f391c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.507+0000 7f3939b09700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 0x7f391c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.507+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 msgr2=0x7f393419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.507+0000 7f3939b09700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39240117e0 tx=0x7f3924011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 shutdown_connections 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f391c038f20 0x7f391c03b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 --2- 172.21.15.132:0/2355791753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934108860 0x7f393419bd10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 >> 172.21.15.132:0/2355791753 conn(0x7f39341022d0 msgr2=0x7f3934102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 shutdown_connections 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:10.508+0000 7f3939b09700 1 -- 172.21.15.132:0/2355791753 wait complete. 2024-03-18T23:00:11.098 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling the dashboard module... 2024-03-18T23:00:11.196 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:10 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:11.196 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:10 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2355791753' entity='client.admin' 2024-03-18T23:00:12.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-mon[32020]: from='mgr.14120 172.21.15.132:0/229107227' entity='mgr.x' 2024-03-18T23:00:12.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1046016681' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-18T23:00:12.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setuser ceph since I am not root 2024-03-18T23:00:12.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setgroup ceph since I am not root 2024-03-18T23:00:12.623 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.484+0000 7f4907ec4700 1 Processor -- start 2024-03-18T23:00:12.623 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.484+0000 7f4907ec4700 1 -- start start 2024-03-18T23:00:12.623 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4907ec4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:12.623 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4907ec4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f490007be20 con 0x7f4900108860 2024-03-18T23:00:12.623 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4905c60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4905c60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49064/0 (socket says 172.21.15.132:49064) 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4905c60700 1 -- 172.21.15.132:0/62231755 learned_addr learned my addr 172.21.15.132:0/62231755 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4905c60700 1 -- 172.21.15.132:0/62231755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490010b190 con 0x7f4900108860 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4905c60700 1 --2- 172.21.15.132:0/62231755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098a0 tx=0x7f48fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e73466eca5e9091 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.485+0000 7f4904c5e700 1 -- 172.21.15.132:0/62231755 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc004160 con 0x7f4900108860 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4904c5e700 1 -- 172.21.15.132:0/62231755 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc004300 con 0x7f4900108860 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4904c5e700 1 -- 172.21.15.132:0/62231755 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc011b20 con 0x7f4900108860 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4907ec4700 1 -- 172.21.15.132:0/62231755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 msgr2=0x7f490010ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4907ec4700 1 --2- 172.21.15.132:0/62231755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098a0 tx=0x7f48fc009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4907ec4700 1 -- 172.21.15.132:0/62231755 shutdown_connections 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4907ec4700 1 --2- 172.21.15.132:0/62231755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490010ac50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.486+0000 7f4907ec4700 1 -- 172.21.15.132:0/62231755 >> 172.21.15.132:0/62231755 conn(0x7f4900102250 msgr2=0x7f4900104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 -- 172.21.15.132:0/62231755 shutdown_connections 2024-03-18T23:00:12.624 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 -- 172.21.15.132:0/62231755 wait complete. 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 Processor -- start 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 -- start start 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4907ec4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f490010c1f0 con 0x7f4900108860 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.487+0000 7f4905c60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4905c60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49078/0 (socket says 172.21.15.132:49078) 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4905c60700 1 -- 172.21.15.132:0/1046016681 learned_addr learned my addr 172.21.15.132:0/1046016681 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4905c60700 1 -- 172.21.15.132:0/1046016681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490019c250 con 0x7f4900108860 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4905c60700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0117e0 tx=0x7f48fc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc003bf0 con 0x7f4900108860 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc01a910 con 0x7f4900108860 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490019c4e0 con 0x7f4900108860 2024-03-18T23:00:12.625 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48fc024970 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.488+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490019c940 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.489+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4900063940 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.489+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f48fc022020 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.489+0000 7f48f6ffd700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 0x7f48ec03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.490+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48fc04d210 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.490+0000 7f490545f700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 0x7f48ec03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.490+0000 7f490545f700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 0x7f48ec03b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.495+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f48fc011e20 con 0x7f4900108860 2024-03-18T23:00:12.626 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:11.764+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f4900106650 con 0x7f4900108860 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.069+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f48fc04cba0 con 0x7f4900108860 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.070+0000 7f48f6ffd700 1 -- 172.21.15.132:0/1046016681 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f48fc02d910 con 0x7f4900108860 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 msgr2=0x7f48ec03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 0x7f48ec03b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 msgr2=0x7f490019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0117e0 tx=0x7f48fc011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 shutdown_connections 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f48ec038ed0 0x7f48ec03b390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 --2- 172.21.15.132:0/1046016681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4900108860 0x7f490019bd10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 >> 172.21.15.132:0/1046016681 conn(0x7f4900102250 msgr2=0x7f4900102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 shutdown_connections 2024-03-18T23:00:12.627 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:12.072+0000 7f4907ec4700 1 -- 172.21.15.132:0/1046016681 wait complete. 2024-03-18T23:00:12.984 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:12.643+0000 7fafa3e3c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:00:12.984 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:12.737+0000 7fafa3e3c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:00:13.241 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1046016681' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-18T23:00:13.241 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-mon[32020]: mgrmap e9: x(active, since 19s) 2024-03-18T23:00:13.512 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:13.388+0000 7fafa3e3c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:00:13.513 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:13.486+0000 7fafa3e3c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:00:13.820 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:13.639+0000 7fafa3e3c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:00:13.820 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:13.737+0000 7fafa3e3c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:00:14.071 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:13.821+0000 7fafa3e3c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:00:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/324695008' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T23:00:14.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:14.096+0000 7fafa3e3c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:00:14.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:14.208+0000 7fafa3e3c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:14.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:14.297+0000 7fafa3e3c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef2e33700 1 Processor -- start 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef2e33700 1 -- start start 2024-03-18T23:00:14.520 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef2e33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef2e33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1076b0 con 0x7f9eec106cc0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef0bcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef0bcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44530/0 (socket says 172.21.15.132:44530) 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef0bcf700 1 -- 172.21.15.132:0/3254217499 learned_addr learned my addr 172.21.15.132:0/3254217499 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.614+0000 7f9ef0bcf700 1 -- 172.21.15.132:0/3254217499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec107ec0 con 0x7f9eec106cc0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9ef0bcf700 1 --2- 172.21.15.132:0/3254217499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00098a0 tx=0x7f9ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b980576c81fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9eeb7fe700 1 -- 172.21.15.132:0/3254217499 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0004160 con 0x7f9eec106cc0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9eeb7fe700 1 -- 172.21.15.132:0/3254217499 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0004300 con 0x7f9eec106cc0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9eeb7fe700 1 -- 172.21.15.132:0/3254217499 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0011b20 con 0x7f9eec106cc0 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9ef2e33700 1 -- 172.21.15.132:0/3254217499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 msgr2=0x7f9eec1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.615+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/3254217499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00098a0 tx=0x7f9ee0009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- 172.21.15.132:0/3254217499 shutdown_connections 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/3254217499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec1070e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.521 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- 172.21.15.132:0/3254217499 >> 172.21.15.132:0/3254217499 conn(0x7f9eec102280 msgr2=0x7f9eec1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- 172.21.15.132:0/3254217499 shutdown_connections 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- 172.21.15.132:0/3254217499 wait complete. 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 Processor -- start 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- start start 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef2e33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10b270 con 0x7f9eec106cc0 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef0bcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef0bcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44536/0 (socket says 172.21.15.132:44536) 2024-03-18T23:00:14.522 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.616+0000 7f9ef0bcf700 1 -- 172.21.15.132:0/324695008 learned_addr learned my addr 172.21.15.132:0/324695008 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ef0bcf700 1 -- 172.21.15.132:0/324695008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec19ea60 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ef0bcf700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00117e0 tx=0x7f9ee0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0003bf0 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec19ec30 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee001a910 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0024970 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.617+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec19efd0 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f9ee0022020 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9ee9ffb700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 0x7f9edc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec0693f0 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ee004d730 con 0x7f9eec106cc0 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9eebfff700 1 -- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 msgr2=0x7f9edc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.618+0000 7f9eebfff700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 0x7f9edc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:00:14.523 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.628+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ee002d910 con 0x7f9eec106cc0 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.819+0000 7f9eebfff700 1 -- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 msgr2=0x7f9edc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.819+0000 7f9eebfff700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 0x7f9edc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.889+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9eec19f2f0 con 0x7f9eec106cc0 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.889+0000 7f9ee9ffb700 1 -- 172.21.15.132:0/324695008 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9ee002d3f0 con 0x7f9eec106cc0 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.891+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 msgr2=0x7f9edc03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.891+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 0x7f9edc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 msgr2=0x7f9eec07ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00117e0 tx=0x7f9ee0011ba0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 shutdown_connections 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f9edc038f70 0x7f9edc03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 --2- 172.21.15.132:0/324695008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eec106cc0 0x7f9eec07ba60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 >> 172.21.15.132:0/324695008 conn(0x7f9eec102280 msgr2=0x7f9eec102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:14.524 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 shutdown_connections 2024-03-18T23:00:14.525 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:13.892+0000 7f9ef2e33700 1 -- 172.21.15.132:0/324695008 wait complete. 2024-03-18T23:00:14.525 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for the mgr to restart... 2024-03-18T23:00:14.525 INFO:teuthology.orchestra.run.smithi132.stdout:Waiting for mgr epoch 9... 2024-03-18T23:00:14.693 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:14.393+0000 7fafa3e3c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:00:14.693 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:14.577+0000 7fafa3e3c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:00:15.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:15.547+0000 7fafa3e3c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:00:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:15.675+0000 7fafa3e3c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:00:16.289 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:16.005+0000 7fafa3e3c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:00:16.289 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:16.094+0000 7fafa3e3c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:00:16.289 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:16.288+0000 7fafa3e3c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:16.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:16.475+0000 7fafa3e3c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:00:17.019 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:16.758+0000 7fafa3e3c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:00:17.019 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:17 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:17.018+0000 7fafa3e3c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:00:17.365 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:17 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:17.253+0000 7fafa3e3c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:00:18.355 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:18.055+0000 7fafa3e3c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:00:18.948 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:18.631+0000 7fafa3e3c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:00:18.948 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:18.731+0000 7fafa3e3c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:00:18.949 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:00:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:00:18.947+0000 7fafa3e3c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: Active manager daemon x restarted 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: Activating manager daemon x 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: osdmap e3: 0 total, 0 up, 0 in 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: mgrmap e10: x(active, starting, since 0.00340311s) 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: Manager daemon x is now available 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:20.182 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:00:20.183 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:00:20.183 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:20 smithi132 ceph-mon[32020]: mgrmap e11: x(active, since 1.00661s) 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout { 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout } 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.895+0000 7f617e8d1700 1 Processor -- start 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.895+0000 7f617e8d1700 1 -- start start 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.895+0000 7f617e8d1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.895+0000 7f617e8d1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f617807be20 con 0x7f6178108870 2024-03-18T23:00:20.894 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6177fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6177fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44544/0 (socket says 172.21.15.132:44544) 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6177fff700 1 -- 172.21.15.132:0/5492912 learned_addr learned my addr 172.21.15.132:0/5492912 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6177fff700 1 -- 172.21.15.132:0/5492912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617810b1a0 con 0x7f6178108870 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6177fff700 1 --2- 172.21.15.132:0/5492912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f616c0098a0 tx=0x7f616c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ccde1456e02daf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6176ffd700 1 -- 172.21.15.132:0/5492912 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616c004160 con 0x7f6178108870 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6176ffd700 1 -- 172.21.15.132:0/5492912 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616c004300 con 0x7f6178108870 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.896+0000 7f6176ffd700 1 -- 172.21.15.132:0/5492912 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616c011b20 con 0x7f6178108870 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 -- 172.21.15.132:0/5492912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 msgr2=0x7f617810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 --2- 172.21.15.132:0/5492912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f616c0098a0 tx=0x7f616c009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 -- 172.21.15.132:0/5492912 shutdown_connections 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 --2- 172.21.15.132:0/5492912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617810ac60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 -- 172.21.15.132:0/5492912 >> 172.21.15.132:0/5492912 conn(0x7f6178102280 msgr2=0x7f61781046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 -- 172.21.15.132:0/5492912 shutdown_connections 2024-03-18T23:00:20.895 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.897+0000 7f617e8d1700 1 -- 172.21.15.132:0/5492912 wait complete. 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f617e8d1700 1 Processor -- start 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f617e8d1700 1 -- start start 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f617e8d1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f617e8d1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f617810c200 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f6177fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f6177fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44548/0 (socket says 172.21.15.132:44548) 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.898+0000 7f6177fff700 1 -- 172.21.15.132:0/1639651774 learned_addr learned my addr 172.21.15.132:0/1639651774 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f6177fff700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617819c210 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f6177fff700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f616c0117e0 tx=0x7f616c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616c003bf0 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616c01a910 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617819c4a0 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f616c024970 con 0x7f6178108870 2024-03-18T23:00:20.896 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.899+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617819c900 con 0x7f6178108870 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f616c022020 con 0x7f6178108870 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f61757fa700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61780693f0 con 0x7f6160038f20 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f616c04d380 con 0x7f6178108870 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f61777fe700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:14.900+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:15.101+0000 7f61777fe700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:15.101+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:15.502+0000 7f61777fe700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:15.502+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:16.302+0000 7f61777fe700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:16.302+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T23:00:20.897 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:17.903+0000 7f61777fe700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3371561746 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:17.903+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:18.963+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f616c02d910 con 0x7f6178108870 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:18.963+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 msgr2=0x7f616003b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:18.963+0000 7f61757fa700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.964+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f616c04e4d0 con 0x7f6178108870 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.964+0000 7f61757fa700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 0x7f616003eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.965+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61780693f0 con 0x7f616003c7b0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.965+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 0x7f616003eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.966+0000 7f61777fe700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 0x7f616003eba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6168003a10 tx=0x7f61680092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.966+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f61780693f0 con 0x7f616003c7b0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6178103430 con 0x7f616003c7b0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f61757fa700 1 -- 172.21.15.132:0/1639651774 <== mgr.14152 v2:172.21.15.132:6800/3730787943 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6178103430 con 0x7f616003c7b0 2024-03-18T23:00:20.898 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 msgr2=0x7f616003eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 0x7f616003eba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6168003a10 tx=0x7f61680092b0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 msgr2=0x7f617819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f616c0117e0 tx=0x7f616c011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 shutdown_connections 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f616003c7b0 0x7f616003eba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178108870 0x7f617819bcd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 --2- 172.21.15.132:0/1639651774 >> [v2:172.21.15.132:6800/3371561746,v1:172.21.15.132:6801/3371561746] conn(0x7f6160038f20 0x7f616003b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 >> 172.21.15.132:0/1639651774 conn(0x7f6178102280 msgr2=0x7f6178102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.973+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 shutdown_connections 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:19.974+0000 7f617e8d1700 1 -- 172.21.15.132:0/1639651774 wait complete. 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:mgr epoch 9 is available 2024-03-18T23:00:20.899 INFO:teuthology.orchestra.run.smithi132.stdout:Generating a dashboard self-signed certificate... 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: [18/Mar/2024:23:00:19] ENGINE Bus STARTING 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: [18/Mar/2024:23:00:19] ENGINE Serving on http://172.21.15.132:8765 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: [18/Mar/2024:23:00:19] ENGINE Serving on https://172.21.15.132:7150 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: [18/Mar/2024:23:00:19] ENGINE Bus STARTED 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T23:00:21.304 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.326+0000 7f0a2973d700 1 Processor -- start 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.326+0000 7f0a2973d700 1 -- start start 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.326+0000 7f0a2973d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.326+0000 7f0a2973d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2407be00 con 0x7f0a241088d0 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a22ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a22ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44592/0 (socket says 172.21.15.132:44592) 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a22ffd700 1 -- 172.21.15.132:0/298386660 learned_addr learned my addr 172.21.15.132:0/298386660 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a22ffd700 1 -- 172.21.15.132:0/298386660 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2410b200 con 0x7f0a241088d0 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a22ffd700 1 --2- 172.21.15.132:0/298386660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e632a409f4984727 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:22.157 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a21ffb700 1 -- 172.21.15.132:0/298386660 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18004160 con 0x7f0a241088d0 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a21ffb700 1 -- 172.21.15.132:0/298386660 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a18004300 con 0x7f0a241088d0 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.327+0000 7f0a21ffb700 1 -- 172.21.15.132:0/298386660 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18011b20 con 0x7f0a241088d0 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 -- 172.21.15.132:0/298386660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 msgr2=0x7f0a2410acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 --2- 172.21.15.132:0/298386660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 -- 172.21.15.132:0/298386660 shutdown_connections 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 --2- 172.21.15.132:0/298386660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a2410acc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 -- 172.21.15.132:0/298386660 >> 172.21.15.132:0/298386660 conn(0x7f0a241022e0 msgr2=0x7f0a24104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 -- 172.21.15.132:0/298386660 shutdown_connections 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 -- 172.21.15.132:0/298386660 wait complete. 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.328+0000 7f0a2973d700 1 Processor -- start 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a2973d700 1 -- start start 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a2973d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:22.158 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a2973d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2410c210 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a22ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a22ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44608/0 (socket says 172.21.15.132:44608) 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a22ffd700 1 -- 172.21.15.132:0/1791791423 learned_addr learned my addr 172.21.15.132:0/1791791423 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a22ffd700 1 -- 172.21.15.132:0/1791791423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a24108690 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a22ffd700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a180117e0 tx=0x7f0a18011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18003bf0 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1801a910 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.329+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a18024970 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.330+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a24106880 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.330+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a24106d40 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.330+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f0a18022020 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.331+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a240693f0 con 0x7f0a241088d0 2024-03-18T23:00:22.159 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.331+0000 7f0a0bfff700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 0x7f0a0c03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.331+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a1804d130 con 0x7f0a241088d0 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.331+0000 7f0a227fc700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 0x7f0a0c03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.331+0000 7f0a227fc700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 0x7f0a0c03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a14006fd0 tx=0x7f0a14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.336+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0a18011e20 con 0x7f0a241088d0 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.532+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f0a18024cd0 con 0x7f0a241088d0 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.547+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0a24107470 con 0x7f0a0c038e10 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.603+0000 7f0a0bfff700 1 -- 172.21.15.132:0/1791791423 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0a24107470 con 0x7f0a0c038e10 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 msgr2=0x7f0a0c03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 0x7f0a0c03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a14006fd0 tx=0x7f0a14006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 msgr2=0x7f0a24108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a180117e0 tx=0x7f0a18011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 shutdown_connections 2024-03-18T23:00:22.160 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0a0c038e10 0x7f0a0c03b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:22.161 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 --2- 172.21.15.132:0/1791791423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a241088d0 0x7f0a24108150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:22.161 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 >> 172.21.15.132:0/1791791423 conn(0x7f0a241022e0 msgr2=0x7f0a24102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:22.161 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 shutdown_connections 2024-03-18T23:00:22.161 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:21.605+0000 7f0a2973d700 1 -- 172.21.15.132:0/1791791423 wait complete. 2024-03-18T23:00:22.161 INFO:teuthology.orchestra.run.smithi132.stdout:Creating initial admin user... 2024-03-18T23:00:22.610 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:22 smithi132 ceph-mon[32020]: mgrmap e12: x(active, since 2s) 2024-03-18T23:00:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:22 smithi132 ceph-mon[32020]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:22.611 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:23.817 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$m1747eXRLPahzZIblt0ZHOL/2f2F8tMIZRwkI5Qsz2VQLceahfDqO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710802823, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bf1e5700 1 Processor -- start 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bf1e5700 1 -- start start 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bf1e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bf1e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f60b807bda0 con 0x7f60b81088f0 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bcf81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bcf81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39176/0 (socket says 172.21.15.132:39176) 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bcf81700 1 -- 172.21.15.132:0/3144300614 learned_addr learned my addr 172.21.15.132:0/3144300614 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.615+0000 7f60bcf81700 1 -- 172.21.15.132:0/3144300614 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b810b220 con 0x7f60b81088f0 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bcf81700 1 --2- 172.21.15.132:0/3144300614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098a0 tx=0x7f60ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8aff76ece7dcba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60b77fe700 1 -- 172.21.15.132:0/3144300614 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60ac004160 con 0x7f60b81088f0 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60b77fe700 1 -- 172.21.15.132:0/3144300614 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ac004300 con 0x7f60b81088f0 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60b77fe700 1 -- 172.21.15.132:0/3144300614 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60ac011b20 con 0x7f60b81088f0 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bf1e5700 1 -- 172.21.15.132:0/3144300614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 msgr2=0x7f60b810ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/3144300614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098a0 tx=0x7f60ac009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bf1e5700 1 -- 172.21.15.132:0/3144300614 shutdown_connections 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/3144300614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b810ace0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.616+0000 7f60bf1e5700 1 -- 172.21.15.132:0/3144300614 >> 172.21.15.132:0/3144300614 conn(0x7f60b8102280 msgr2=0x7f60b81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 -- 172.21.15.132:0/3144300614 shutdown_connections 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 -- 172.21.15.132:0/3144300614 wait complete. 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 Processor -- start 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 -- start start 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bf1e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810c1d0 con 0x7f60b81088f0 2024-03-18T23:00:23.819 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bcf81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bcf81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39186/0 (socket says 172.21.15.132:39186) 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.617+0000 7f60bcf81700 1 -- 172.21.15.132:0/2593842982 learned_addr learned my addr 172.21.15.132:0/2593842982 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60bcf81700 1 -- 172.21.15.132:0/2593842982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b819c210 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60bcf81700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0117e0 tx=0x7f60ac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60ac003bf0 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ac01a910 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b819c4a0 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60ac024970 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.618+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b819c900 con 0x7f60b81088f0 2024-03-18T23:00:23.820 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.619+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f60ac022020 con 0x7f60b81088f0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.619+0000 7f60b5ffb700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 0x7f60a803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.619+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f60ac04d260 con 0x7f60b81088f0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.619+0000 7f60b7fff700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 0x7f60a803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.619+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b80693f0 con 0x7f60b81088f0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.620+0000 7f60b7fff700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 0x7f60a803b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60a4006fd0 tx=0x7f60a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.624+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f60ac011e20 con 0x7f60b81088f0 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:22.850+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f60b819f240 con 0x7f60a8038f80 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.075+0000 7f60b5ffb700 1 -- 172.21.15.132:0/2593842982 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f60b819f240 con 0x7f60a8038f80 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 msgr2=0x7f60a803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:23.821 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 0x7f60a803b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60a4006fd0 tx=0x7f60a4006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 msgr2=0x7f60b819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0117e0 tx=0x7f60ac011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 shutdown_connections 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f60a8038f80 0x7f60a803b440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 --2- 172.21.15.132:0/2593842982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81088f0 0x7f60b819bcd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 >> 172.21.15.132:0/2593842982 conn(0x7f60b8102280 msgr2=0x7f60b8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 shutdown_connections 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:23.078+0000 7f60bf1e5700 1 -- 172.21.15.132:0/2593842982 wait complete. 2024-03-18T23:00:23.822 INFO:teuthology.orchestra.run.smithi132.stdout:Fetching dashboard port number... 2024-03-18T23:00:24.357 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:24 smithi132 ceph-mon[32020]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:24.357 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:24 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:25.186 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stdout 8443 2024-03-18T23:00:25.186 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.229+0000 7f0726a5c700 1 Processor -- start 2024-03-18T23:00:25.186 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.229+0000 7f0726a5c700 1 -- start start 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f0726a5c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f0726a5c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0720108780 con 0x7f0720107d90 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39198/0 (socket says 172.21.15.132:39198) 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071ffff700 1 -- 172.21.15.132:0/2519016284 learned_addr learned my addr 172.21.15.132:0/2519016284 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071ffff700 1 -- 172.21.15.132:0/2519016284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720108f90 con 0x7f0720107d90 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071ffff700 1 --2- 172.21.15.132:0/2519016284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07100098a0 tx=0x7f0710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964cb3a5b21c45b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071effd700 1 -- 172.21.15.132:0/2519016284 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0710004160 con 0x7f0720107d90 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071effd700 1 -- 172.21.15.132:0/2519016284 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710004300 con 0x7f0720107d90 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.230+0000 7f071effd700 1 -- 172.21.15.132:0/2519016284 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0710011b20 con 0x7f0720107d90 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 -- 172.21.15.132:0/2519016284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 msgr2=0x7f07201081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 --2- 172.21.15.132:0/2519016284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07100098a0 tx=0x7f0710009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:25.187 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 -- 172.21.15.132:0/2519016284 shutdown_connections 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 --2- 172.21.15.132:0/2519016284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f07201081b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 -- 172.21.15.132:0/2519016284 >> 172.21.15.132:0/2519016284 conn(0x7f0720103330 msgr2=0x7f0720105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 -- 172.21.15.132:0/2519016284 shutdown_connections 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.231+0000 7f0726a5c700 1 -- 172.21.15.132:0/2519016284 wait complete. 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f0726a5c700 1 Processor -- start 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f0726a5c700 1 -- start start 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f0726a5c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f0726a5c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f072010c340 con 0x7f0720107d90 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f071ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f071ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39208/0 (socket says 172.21.15.132:39208) 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.232+0000 7f071ffff700 1 -- 172.21.15.132:0/1234151941 learned_addr learned my addr 172.21.15.132:0/1234151941 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f071ffff700 1 -- 172.21.15.132:0/1234151941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072019c260 con 0x7f0720107d90 2024-03-18T23:00:25.188 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f071ffff700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07100117e0 tx=0x7f0710011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0710003bf0 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072019c4f0 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072019c950 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f071001a910 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.233+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0710024970 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.234+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f0710022020 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.234+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c005360 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.234+0000 7f071d7fa700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 0x7f070803b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.234+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f071004e710 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.234+0000 7f071f7fe700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 0x7f070803b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.235+0000 7f071f7fe700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 0x7f070803b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0714006fd0 tx=0x7f0714006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.239+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07100172b0 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.442+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f070c005650 con 0x7f0720107d90 2024-03-18T23:00:25.189 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.443+0000 7f071d7fa700 1 -- 172.21.15.132:0/1234151941 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f0710018350 con 0x7f0720107d90 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 msgr2=0x7f070803b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 0x7f070803b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0714006fd0 tx=0x7f0714006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 msgr2=0x7f072019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07100117e0 tx=0x7f0710011ba0 comp rx=0 tx=0).stop 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 shutdown_connections 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0708038fd0 0x7f070803b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 --2- 172.21.15.132:0/1234151941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0720107d90 0x7f072019bd20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 >> 172.21.15.132:0/1234151941 conn(0x7f0720103330 msgr2=0x7f0720103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 shutdown_connections 2024-03-18T23:00:25.190 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:24.445+0000 7f0726a5c700 1 -- 172.21.15.132:0/1234151941 wait complete. 2024-03-18T23:00:25.194 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T23:00:25.194 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout disabled 2024-03-18T23:00:25.202 INFO:teuthology.orchestra.run.smithi132.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T23:00:25.202 INFO:teuthology.orchestra.run.smithi132.stdout:systemctl: stdout inactive 2024-03-18T23:00:25.202 INFO:teuthology.orchestra.run.smithi132.stdout:firewalld.service is not enabled 2024-03-18T23:00:25.202 INFO:teuthology.orchestra.run.smithi132.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout:Ceph Dashboard is now available at: 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout: URL: https://smithi132.front.sepia.ceph.com:8443/ 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout: User: admin 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout: Password: b3uh5d8qhw 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout:Saving cluster configuration to /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config directory 2024-03-18T23:00:25.204 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling autotune for osd_memory_target 2024-03-18T23:00:25.320 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:25 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1234151941' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-18T23:00:26.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:26 smithi132 ceph-mon[32020]: mgrmap e13: x(active, since 6s) 2024-03-18T23:00:26.424 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.611+0000 7f25affdd700 1 Processor -- start 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.611+0000 7f25affdd700 1 -- start start 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25affdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25affdd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f25a807bde0 con 0x7f25a8106690 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25add79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25add79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39224/0 (socket says 172.21.15.132:39224) 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25add79700 1 -- 172.21.15.132:0/4069721487 learned_addr learned my addr 172.21.15.132:0/4069721487 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25add79700 1 -- 172.21.15.132:0/4069721487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a8106ff0 con 0x7f25a8106690 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25add79700 1 --2- 172.21.15.132:0/4069721487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25980098a0 tx=0x7f2598009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13564f8bc1a7f7f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25acd77700 1 -- 172.21.15.132:0/4069721487 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2598004160 con 0x7f25a8106690 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25acd77700 1 -- 172.21.15.132:0/4069721487 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2598004300 con 0x7f25a8106690 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.612+0000 7f25acd77700 1 -- 172.21.15.132:0/4069721487 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2598011b20 con 0x7f25a8106690 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 -- 172.21.15.132:0/4069721487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 msgr2=0x7f25a8106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:26.425 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 --2- 172.21.15.132:0/4069721487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25980098a0 tx=0x7f2598009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 -- 172.21.15.132:0/4069721487 shutdown_connections 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 --2- 172.21.15.132:0/4069721487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a8106ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 -- 172.21.15.132:0/4069721487 >> 172.21.15.132:0/4069721487 conn(0x7f25a81022d0 msgr2=0x7f25a81046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 -- 172.21.15.132:0/4069721487 shutdown_connections 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.613+0000 7f25affdd700 1 -- 172.21.15.132:0/4069721487 wait complete. 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 Processor -- start 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 -- start start 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f25a81080c0 con 0x7f25a8106690 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25add79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25add79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39232/0 (socket says 172.21.15.132:39232) 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25add79700 1 -- 172.21.15.132:0/2971778402 learned_addr learned my addr 172.21.15.132:0/2971778402 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25add79700 1 -- 172.21.15.132:0/2971778402 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a819e450 con 0x7f25a8106690 2024-03-18T23:00:26.426 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25add79700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25980117e0 tx=0x7f2598011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2598003bf0 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f259801a910 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a819e6e0 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2598024970 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.614+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a819eb40 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.615+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a8063940 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.615+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f2598022020 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.615+0000 7f259effd700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 0x7f259403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.615+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f259804d3c0 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.616+0000 7f25ad578700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 0x7f259403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.616+0000 7f25ad578700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 0x7f259403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006fd0 tx=0x7f25a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.620+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2598011e20 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.814+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f25a8062f20 con 0x7f25a8106690 2024-03-18T23:00:26.427 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.815+0000 7f259effd700 1 -- 172.21.15.132:0/2971778402 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f259802ddb0 con 0x7f25a8106690 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 msgr2=0x7f259403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 0x7f259403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006fd0 tx=0x7f25a4006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 msgr2=0x7f25a819df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25980117e0 tx=0x7f2598011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 shutdown_connections 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2594038fc0 0x7f259403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 --2- 172.21.15.132:0/2971778402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25a8106690 0x7f25a819df10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 >> 172.21.15.132:0/2971778402 conn(0x7f25a81022d0 msgr2=0x7f25a8194bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 shutdown_connections 2024-03-18T23:00:26.428 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:25.817+0000 7f25affdd700 1 -- 172.21.15.132:0/2971778402 wait complete. 2024-03-18T23:00:27.674 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.812+0000 7f3e22db0700 1 Processor -- start 2024-03-18T23:00:27.674 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.812+0000 7f3e22db0700 1 -- start start 2024-03-18T23:00:27.674 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.812+0000 7f3e22db0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:27.674 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.812+0000 7f3e22db0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c106ab0 con 0x7f3e1c1060c0 2024-03-18T23:00:27.674 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e20b4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e20b4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39236/0 (socket says 172.21.15.132:39236) 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e20b4c700 1 -- 172.21.15.132:0/1265694154 learned_addr learned my addr 172.21.15.132:0/1265694154 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e20b4c700 1 -- 172.21.15.132:0/1265694154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c109050 con 0x7f3e1c1060c0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e20b4c700 1 --2- 172.21.15.132:0/1265694154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e100098a0 tx=0x7f3e10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c3146cfddb09da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e1b7fe700 1 -- 172.21.15.132:0/1265694154 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e10004160 con 0x7f3e1c1060c0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e1b7fe700 1 -- 172.21.15.132:0/1265694154 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10004300 con 0x7f3e1c1060c0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.813+0000 7f3e1b7fe700 1 -- 172.21.15.132:0/1265694154 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e10011b20 con 0x7f3e1c1060c0 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- 172.21.15.132:0/1265694154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 msgr2=0x7f3e1c1064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 --2- 172.21.15.132:0/1265694154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e100098a0 tx=0x7f3e10009bb0 comp rx=0 tx=0).stop 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- 172.21.15.132:0/1265694154 shutdown_connections 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 --2- 172.21.15.132:0/1265694154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1064e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- 172.21.15.132:0/1265694154 >> 172.21.15.132:0/1265694154 conn(0x7f3e1c101a10 msgr2=0x7f3e1c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:27.675 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- 172.21.15.132:0/1265694154 shutdown_connections 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- 172.21.15.132:0/1265694154 wait complete. 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 Processor -- start 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.814+0000 7f3e22db0700 1 -- start start 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e22db0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e22db0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c10c400 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e20b4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e20b4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39250/0 (socket says 172.21.15.132:39250) 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e20b4c700 1 -- 172.21.15.132:0/3221028057 learned_addr learned my addr 172.21.15.132:0/3221028057 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e20b4c700 1 -- 172.21.15.132:0/3221028057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c108e10 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e20b4c700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e100117e0 tx=0x7f3e10011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e10003bf0 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1001a910 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c107000 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.815+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e100249a0 con 0x7f3e1c1060c0 2024-03-18T23:00:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.816+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1074c0 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.816+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f3e10022020 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.816+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c063940 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.817+0000 7f3e19ffb700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 0x7f3e0c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.817+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e1004d520 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.817+0000 7f3e1bfff700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 0x7f3e0c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.817+0000 7f3e1bfff700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 0x7f3e0c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e08006fd0 tx=0x7f3e08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:26.822+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e10011e20 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.126+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3e1c107b80 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.128+0000 7f3e19ffb700 1 -- 172.21.15.132:0/3221028057 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3e1000ba00 con 0x7f3e1c1060c0 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 msgr2=0x7f3e0c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 0x7f3e0c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3e08006fd0 tx=0x7f3e08006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 msgr2=0x7f3e1c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3e100117e0 tx=0x7f3e10011c10 comp rx=0 tx=0).stop 2024-03-18T23:00:27.677 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 shutdown_connections 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3e0c038fc0 0x7f3e0c03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 --2- 172.21.15.132:0/3221028057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c1060c0 0x7f3e1c1088d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 >> 172.21.15.132:0/3221028057 conn(0x7f3e1c101a10 msgr2=0x7f3e1c1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 shutdown_connections 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr 2024-03-18T23:00:27.130+0000 7f3e22db0700 1 -- 172.21.15.132:0/3221028057 wait complete. 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout:Or, if you are only running a single cluster on this host: 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-18T23:00:27.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: ceph telemetry on 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout:For more information see: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:27.679 INFO:teuthology.orchestra.run.smithi132.stdout:Bootstrap complete. 2024-03-18T23:00:27.711 INFO:tasks.cephadm:Fetching config... 2024-03-18T23:00:27.712 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:00:27.712 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-18T23:00:27.727 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-18T23:00:27.727 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:00:27.727 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-18T23:00:27.782 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-18T23:00:27.782 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:00:27.783 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-18T23:00:27.849 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-18T23:00:27.849 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:00:27.849 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-18T23:00:27.905 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-18T23:00:27.905 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T23:00:27.983 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:00:27.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T23:00:28.046 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:00:28.062 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T23:00:28.110 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXtSTvGr/QrdLsFB/XHg+MyuAKYze4obxmkfOtb+gH+oXY2X3yvyDXkKtm235xZHGAml84yKthQYowrl9PNe3LXC+t6ICq0zUCjKC1gQbfm9bjE8ESKdaK8HZxQ6DDfsH8z61uPz3qdrrvAgqPJ1i/lOU5WQ9uJd521FBBGFooeQebf7je7ScsVJuz+87avyYVAR5ulJdrc8ytqRdClNTas1O6lKm44iiBsKUtlpo7jgh4KDijStU3/1pkujz818iFJTNpgAzoG1Qd9Qa4JiFfhoHE15bIl7cCyaS4M8PrEq8pQW6QUFabWswG65ZksjNWtAcIDMiAyJSlu4UQtTozYCl7OlzESussgNCBTiXIByr5chx4cTlsyX8ySp2pO2W7L7/RHcxEf5bnniw3qE6NPRaBR4JSjNQKSiAIkLbK8Bv0bIX4RidSuLFgIrOXYsITkBZhAAocGiQA1JL1Qhp3Cpcfut2UTg18e6YQFPqvMWMAnTOHXt75Pk2IB6FYiYs= ceph-0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:00:28.125 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-18T23:00:28.365 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:28 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3221028057' entity='client.admin' 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 -- 172.21.15.132:0/2899990629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 msgr2=0x7f00e41037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 --2- 172.21.15.132:0/2899990629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41037e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009910 tx=0x7f00d4009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 -- 172.21.15.132:0/2899990629 shutdown_connections 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 --2- 172.21.15.132:0/2899990629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41037e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 -- 172.21.15.132:0/2899990629 >> 172.21.15.132:0/2899990629 conn(0x7f00e40fb9c0 msgr2=0x7f00e40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:30.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.852+0000 7f00e98be700 1 -- 172.21.15.132:0/2899990629 shutdown_connections 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.853+0000 7f00e98be700 1 -- 172.21.15.132:0/2899990629 wait complete. 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.853+0000 7f00e98be700 1 Processor -- start 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.853+0000 7f00e98be700 1 -- start start 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.853+0000 7f00e98be700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.853+0000 7f00e98be700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4107d60 con 0x7f00e41033c0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39272/0 (socket says 172.21.15.132:39272) 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e2ffd700 1 -- 172.21.15.132:0/545551559 learned_addr learned my addr 172.21.15.132:0/545551559 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e2ffd700 1 -- 172.21.15.132:0/545551559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e4197920 con 0x7f00e41033c0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e2ffd700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d4027070 con 0x7f00e41033c0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e98be700 1 -- 172.21.15.132:0/545551559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e4197c10 con 0x7f00e41033c0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d400be40 con 0x7f00e41033c0 2024-03-18T23:00:30.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.854+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00d400f460 con 0x7f00e41033c0 2024-03-18T23:00:30.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.855+0000 7f00e98be700 1 -- 172.21.15.132:0/545551559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e4100f50 con 0x7f00e41033c0 2024-03-18T23:00:30.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.856+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e41084e0 con 0x7f00e41033c0 2024-03-18T23:00:30.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.856+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f00d402a070 con 0x7f00e41033c0 2024-03-18T23:00:30.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.856+0000 7f00e88bc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 0x7f00cc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:30.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.856+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00d401e030 con 0x7f00e41033c0 2024-03-18T23:00:30.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.857+0000 7f00e27fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 0x7f00cc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:30.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.857+0000 7f00e27fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 0x7f00cc03b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006fd0 tx=0x7f00d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:30.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:30.861+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f00d4031070 con 0x7f00e41033c0 2024-03-18T23:00:31.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.064+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f00e4102cf0 con 0x7f00e41033c0 2024-03-18T23:00:31.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.068+0000 7f00e88bc700 1 -- 172.21.15.132:0/545551559 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f00d4013070 con 0x7f00e41033c0 2024-03-18T23:00:31.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 msgr2=0x7f00cc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:31.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 0x7f00cc03b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f00d8006fd0 tx=0x7f00d8006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:31.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 msgr2=0x7f00e41973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:31.077 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4005050 comp rx=0 tx=0).stop 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 shutdown_connections 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f00cc039020 0x7f00cc03b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 --2- 172.21.15.132:0/545551559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00e41033c0 0x7f00e41973e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.071+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 >> 172.21.15.132:0/545551559 conn(0x7f00e40fb9c0 msgr2=0x7f00e40fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.072+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 shutdown_connections 2024-03-18T23:00:31.078 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:31.072+0000 7f00ca7fc700 1 -- 172.21.15.132:0/545551559 wait complete. 2024-03-18T23:00:31.894 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-18T23:00:31.894 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-18T23:00:32.294 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:32 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/545551559' entity='client.admin' 2024-03-18T23:00:32.367 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:33.596 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2429348088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 msgr2=0x7fc6d01029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2429348088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d01029f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009910 tx=0x7fc6cc009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2429348088 shutdown_connections 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2429348088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d01029f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2429348088 >> 172.21.15.132:0/2429348088 conn(0x7fc6d00fc1e0 msgr2=0x7fc6d00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2429348088 shutdown_connections 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2429348088 wait complete. 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 Processor -- start 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.809+0000 7fc6d7f8e700 1 -- start start 2024-03-18T23:00:33.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d7f8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d7f8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0105c80 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d5d2a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d5d2a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41790/0 (socket says 172.21.15.132:41790) 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d5d2a700 1 -- 172.21.15.132:0/2531138966 learned_addr learned my addr 172.21.15.132:0/2531138966 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d5d2a700 1 -- 172.21.15.132:0/2531138966 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d019df70 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d5d2a700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc004fa0 tx=0x7fc6cc005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc027070 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc00be40 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6cc00f460 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d019e200 con 0x7fc6d01005c0 2024-03-18T23:00:33.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.810+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a0ef0 con 0x7fc6d01005c0 2024-03-18T23:00:33.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.811+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fc6cc02a070 con 0x7fc6d01005c0 2024-03-18T23:00:33.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.811+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d0106400 con 0x7fc6d01005c0 2024-03-18T23:00:33.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.811+0000 7fc6c6ffd700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 0x7fc6bc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:33.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.811+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6cc01e030 con 0x7fc6d01005c0 2024-03-18T23:00:33.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.811+0000 7fc6d5529700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 0x7fc6bc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:33.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.812+0000 7fc6d5529700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 0x7fc6bc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:33.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:33.816+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6cc013070 con 0x7fc6d01005c0 2024-03-18T23:00:34.016 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.015+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc6d002dcc0 con 0x7fc6bc039020 2024-03-18T23:00:34.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.018+0000 7fc6c6ffd700 1 -- 172.21.15.132:0/2531138966 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6d002dcc0 con 0x7fc6bc039020 2024-03-18T23:00:34.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.020+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 msgr2=0x7fc6bc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.020+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 0x7fc6bc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.020+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 msgr2=0x7fc6d019da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.020+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc004fa0 tx=0x7fc6cc005080 comp rx=0 tx=0).stop 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 shutdown_connections 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc6bc039020 0x7fc6bc03b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 --2- 172.21.15.132:0/2531138966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6d01005c0 0x7fc6d019da30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 >> 172.21.15.132:0/2531138966 conn(0x7fc6d00fc1e0 msgr2=0x7fc6d00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 shutdown_connections 2024-03-18T23:00:34.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:34.021+0000 7fc6d7f8e700 1 -- 172.21.15.132:0/2531138966 wait complete. 2024-03-18T23:00:34.774 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-03-18T23:00:34.774 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T23:00:34.774 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-03-18T23:00:34.793 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T23:00:34.793 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:34.850 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-03-18T23:00:34.850 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch host add smithi144 2024-03-18T23:00:35.088 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:00:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:36.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:36 smithi132 ceph-mon[32020]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:36.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:36 smithi132 ceph-mon[32020]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:36.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:36 smithi132 ceph-mon[32020]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- 172.21.15.132:0/3402818840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 msgr2=0x7f0664101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 --2- 172.21.15.132:0/3402818840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f0664101220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0654009910 tx=0x7f0654009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- 172.21.15.132:0/3402818840 shutdown_connections 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 --2- 172.21.15.132:0/3402818840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f0664101220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- 172.21.15.132:0/3402818840 >> 172.21.15.132:0/3402818840 conn(0x7f06640fc710 msgr2=0x7f06640feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- 172.21.15.132:0/3402818840 shutdown_connections 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- 172.21.15.132:0/3402818840 wait complete. 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 Processor -- start 2024-03-18T23:00:36.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.596+0000 7f066a5e0700 1 -- start start 2024-03-18T23:00:36.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f066a5e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:36.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f066a5e0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f066410b210 con 0x7f0664100e40 2024-03-18T23:00:36.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f0663fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:36.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f0663fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41816/0 (socket says 172.21.15.132:41816) 2024-03-18T23:00:36.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f0663fff700 1 -- 172.21.15.132:0/594007877 learned_addr learned my addr 172.21.15.132:0/594007877 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f0663fff700 1 -- 172.21.15.132:0/594007877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066419d9a0 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f0663fff700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0654005850 tx=0x7f0654005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0654026070 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065400bbd0 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066419dc90 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0654003970 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.597+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0664197620 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.598+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f0654029070 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.598+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0664102320 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.598+0000 7f06617fa700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 0x7f064c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.598+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f065401e070 con 0x7f0664100e40 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.599+0000 7f06637fe700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 0x7f064c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:36.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.599+0000 7f06637fe700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 0x7f064c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0658006fd0 tx=0x7f0658006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.603+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0654013070 con 0x7f0664100e40 2024-03-18T23:00:36.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:36.803+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f0664198b50 con 0x7f064c038fc0 2024-03-18T23:00:37.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:37 smithi132 ceph-mon[32020]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:38.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:38 smithi132 ceph-mon[32020]: Deploying cephadm binary to smithi144 2024-03-18T23:00:40.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:39 smithi132 ceph-mon[32020]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:41.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.067+0000 7f06617fa700 1 -- 172.21.15.132:0/594007877 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0664198b50 con 0x7f064c038fc0 2024-03-18T23:00:41.068 INFO:teuthology.orchestra.run.smithi132.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-03-18T23:00:41.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 msgr2=0x7f064c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:41.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 0x7f064c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0658006fd0 tx=0x7f0658006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 msgr2=0x7f066419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0654005850 tx=0x7f0654005030 comp rx=0 tx=0).stop 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 shutdown_connections 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f064c038fc0 0x7f064c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 --2- 172.21.15.132:0/594007877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0664100e40 0x7f066419d460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 >> 172.21.15.132:0/594007877 conn(0x7f06640fc710 msgr2=0x7f066406a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 shutdown_connections 2024-03-18T23:00:41.071 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:41.070+0000 7f066a5e0700 1 -- 172.21.15.132:0/594007877 wait complete. 2024-03-18T23:00:41.682 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T23:00:41.923 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:42.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:42.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:42.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: Added host smithi144 2024-03-18T23:00:42.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:42.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:42.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 -- 172.21.15.132:0/1986383226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 msgr2=0x7f000c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 --2- 172.21.15.132:0/1986383226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c102310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efffc009910 tx=0x7efffc009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 -- 172.21.15.132:0/1986383226 shutdown_connections 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 --2- 172.21.15.132:0/1986383226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c102310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 -- 172.21.15.132:0/1986383226 >> 172.21.15.132:0/1986383226 conn(0x7f000c0fd7e0 msgr2=0x7f000c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 -- 172.21.15.132:0/1986383226 shutdown_connections 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.432+0000 7f001177f700 1 -- 172.21.15.132:0/1986383226 wait complete. 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f001177f700 1 Processor -- start 2024-03-18T23:00:43.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f001177f700 1 -- start start 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f001177f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f001177f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f000c10c300 con 0x7f000c101f30 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f000affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f000affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59676/0 (socket says 172.21.15.132:59676) 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f000affd700 1 -- 172.21.15.132:0/2615041395 learned_addr learned my addr 172.21.15.132:0/2615041395 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:43.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.433+0000 7f000affd700 1 -- 172.21.15.132:0/2615041395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c19daa0 con 0x7f000c101f30 2024-03-18T23:00:43.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7f000affd700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efffc005850 tx=0x7efffc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:43.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efffc026070 con 0x7f000c101f30 2024-03-18T23:00:43.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc00bbd0 con 0x7f000c101f30 2024-03-18T23:00:43.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c19dd90 con 0x7f000c101f30 2024-03-18T23:00:43.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efffc003970 con 0x7f000c101f30 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.434+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c197800 con 0x7f000c101f30 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7efffc029070 con 0x7f000c101f30 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7efff3fff700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 0x7efff403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efffc01e070 con 0x7f000c101f30 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7f000a7fc700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 0x7efff403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:43.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c103410 con 0x7f000c101f30 2024-03-18T23:00:43.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.435+0000 7f000a7fc700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 0x7efff403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0000006fd0 tx=0x7f0000006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:43.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.440+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efffc030070 con 0x7f000c101f30 2024-03-18T23:00:43.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.640+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f000c198b60 con 0x7efff4038fc0 2024-03-18T23:00:43.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.642+0000 7efff3fff700 1 -- 172.21.15.132:0/2615041395 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f000c198b60 con 0x7efff4038fc0 2024-03-18T23:00:43.642 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:43.642 INFO:teuthology.orchestra.run.smithi132.stdout:[{"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-03-18T23:00:43.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 msgr2=0x7efff403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:43.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 0x7efff403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0000006fd0 tx=0x7f0000006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 msgr2=0x7f000c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efffc005850 tx=0x7efffc005030 comp rx=0 tx=0).stop 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 shutdown_connections 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efff4038fc0 0x7efff403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 --2- 172.21.15.132:0/2615041395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c101f30 0x7f000c19d560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 >> 172.21.15.132:0/2615041395 conn(0x7f000c0fd7e0 msgr2=0x7f000c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 shutdown_connections 2024-03-18T23:00:43.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:43.644+0000 7f001177f700 1 -- 172.21.15.132:0/2615041395 wait complete. 2024-03-18T23:00:44.278 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:44 smithi132 ceph-mon[32020]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:44.278 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:44.306 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-03-18T23:00:44.307 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T23:00:44.307 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-03-18T23:00:44.324 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T23:00:44.325 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:44.381 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-03-18T23:00:44.382 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch host add smithi196 2024-03-18T23:00:44.619 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:45.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:45 smithi132 ceph-mon[32020]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:00:46.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 -- 172.21.15.132:0/2938741069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 msgr2=0x7f1734102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 --2- 172.21.15.132:0/2938741069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f1734102310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f172c00b0a0 tx=0x7f172c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 -- 172.21.15.132:0/2938741069 shutdown_connections 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 --2- 172.21.15.132:0/2938741069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f1734102310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 -- 172.21.15.132:0/2938741069 >> 172.21.15.132:0/2938741069 conn(0x7f17340fd7e0 msgr2=0x7f17340ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 -- 172.21.15.132:0/2938741069 shutdown_connections 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.064+0000 7f173aba7700 1 -- 172.21.15.132:0/2938741069 wait complete. 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f173aba7700 1 Processor -- start 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f173aba7700 1 -- start start 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f173aba7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f173aba7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f173410c300 con 0x7f1734101f30 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f1738943700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:46.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f1738943700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59698/0 (socket says 172.21.15.132:59698) 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f1738943700 1 -- 172.21.15.132:0/2635741788 learned_addr learned my addr 172.21.15.132:0/2635741788 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.065+0000 7f1738943700 1 -- 172.21.15.132:0/2635741788 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173419d970 con 0x7f1734101f30 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f1738943700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f172c015040 tx=0x7f172c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c017040 con 0x7f1734101f30 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173419dc00 con 0x7f1734101f30 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c022070 con 0x7f1734101f30 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c01d5d0 con 0x7f1734101f30 2024-03-18T23:00:46.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.066+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341975f0 con 0x7f1734101f30 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.067+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f172c004000 con 0x7f1734101f30 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.067+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1734103410 con 0x7f1734101f30 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.067+0000 7f1731ffb700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 0x7f172003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.067+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f172c00f050 con 0x7f1734101f30 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.067+0000 7f1733fff700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 0x7f172003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:46.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.068+0000 7f1733fff700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 0x7f172003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1724006fd0 tx=0x7f1724006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:46.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.072+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f172c007e20 con 0x7f1734101f30 2024-03-18T23:00:46.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:46.277+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f1734063940 con 0x7f1720038fc0 2024-03-18T23:00:46.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:46 smithi132 ceph-mon[32020]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:47.454 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:47.455 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: Deploying cephadm binary to smithi196 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.456 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:47.457 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:50 smithi132 ceph-mon[32020]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:50.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.445+0000 7f1731ffb700 1 -- 172.21.15.132:0/2635741788 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1734063940 con 0x7f1720038fc0 2024-03-18T23:00:50.446 INFO:teuthology.orchestra.run.smithi132.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 msgr2=0x7f172003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 0x7f172003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1724006fd0 tx=0x7f1724006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 msgr2=0x7f173419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f172c015040 tx=0x7f172c003d50 comp rx=0 tx=0).stop 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 shutdown_connections 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1720038fc0 0x7f172003b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 --2- 172.21.15.132:0/2635741788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1734101f30 0x7f173419d430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:50.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 >> 172.21.15.132:0/2635741788 conn(0x7f17340fd7e0 msgr2=0x7f1734104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:50.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.447+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 shutdown_connections 2024-03-18T23:00:50.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:50.448+0000 7f173aba7700 1 -- 172.21.15.132:0/2635741788 wait complete. 2024-03-18T23:00:51.081 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T23:00:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:51.719 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:51 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:51.719 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:51 smithi132 ceph-mon[32020]: Added host smithi196 2024-03-18T23:00:51.719 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:51 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:51.719 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:51 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:51.719 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:51 smithi132 ceph-mon[32020]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:52.519 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:52 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 -- 172.21.15.132:0/2328335217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 msgr2=0x7fadac102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 --2- 172.21.15.132:0/2328335217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac102310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fada0009910 tx=0x7fada0009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 -- 172.21.15.132:0/2328335217 shutdown_connections 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 --2- 172.21.15.132:0/2328335217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac102310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 -- 172.21.15.132:0/2328335217 >> 172.21.15.132:0/2328335217 conn(0x7fadac0fd7e0 msgr2=0x7fadac0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 -- 172.21.15.132:0/2328335217 shutdown_connections 2024-03-18T23:00:52.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.742+0000 7fadb0dba700 1 -- 172.21.15.132:0/2328335217 wait complete. 2024-03-18T23:00:52.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadb0dba700 1 Processor -- start 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadb0dba700 1 -- start start 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadb0dba700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadb0dba700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10c300 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadaa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadaa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58566/0 (socket says 172.21.15.132:58566) 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadaa59c700 1 -- 172.21.15.132:0/517228095 learned_addr learned my addr 172.21.15.132:0/517228095 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadaa59c700 1 -- 172.21.15.132:0/517228095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac076500 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fadaa59c700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fada0005850 tx=0x7fada0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.743+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0026070 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.744+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac074bb0 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.744+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada000bc30 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.744+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0003a20 con 0x7fadac101f30 2024-03-18T23:00:52.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.744+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac075070 con 0x7fadac101f30 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.744+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fada0029070 con 0x7fadac101f30 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.745+0000 7fad9b7fe700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 0x7fad9403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.745+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fada001e070 con 0x7fadac101f30 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.745+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac103410 con 0x7fadac101f30 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.745+0000 7fada9d9b700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 0x7fad9403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:52.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.745+0000 7fada9d9b700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 0x7fad9403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006fd0 tx=0x7fad9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:52.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.751+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fada0030070 con 0x7fadac101f30 2024-03-18T23:00:52.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.950+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadac075b40 con 0x7fad94038fc0 2024-03-18T23:00:52.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.951+0000 7fad9b7fe700 1 -- 172.21.15.132:0/517228095 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fadac075b40 con 0x7fad94038fc0 2024-03-18T23:00:52.952 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:00:52.952 INFO:teuthology.orchestra.run.smithi132.stdout:[{"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-03-18T23:00:52.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 msgr2=0x7fad9403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 0x7fad9403b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006fd0 tx=0x7fad9c006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 msgr2=0x7fadac074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fada0005850 tx=0x7fada0005030 comp rx=0 tx=0).stop 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 shutdown_connections 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fad94038fc0 0x7fad9403b480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 --2- 172.21.15.132:0/517228095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadac101f30 0x7fadac074610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 >> 172.21.15.132:0/517228095 conn(0x7fadac0fd7e0 msgr2=0x7fadac104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 shutdown_connections 2024-03-18T23:00:52.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:52.954+0000 7fadb0dba700 1 -- 172.21.15.132:0/517228095 wait complete. 2024-03-18T23:00:53.548 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-18T23:00:53.548 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-18T23:00:53.780 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:00:54.214 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:54 smithi132 ceph-mon[32020]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:00:54.215 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:54 smithi132 ceph-mon[32020]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:54.215 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:54 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:55.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 -- 172.21.15.132:0/923360909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 msgr2=0x7fe19c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/923360909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c102310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe190009910 tx=0x7fe190009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 -- 172.21.15.132:0/923360909 shutdown_connections 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/923360909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c102310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 -- 172.21.15.132:0/923360909 >> 172.21.15.132:0/923360909 conn(0x7fe19c0fd7e0 msgr2=0x7fe19c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 -- 172.21.15.132:0/923360909 shutdown_connections 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.164+0000 7fe1a30ff700 1 -- 172.21.15.132:0/923360909 wait complete. 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a30ff700 1 Processor -- start 2024-03-18T23:00:55.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a30ff700 1 -- start start 2024-03-18T23:00:55.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a30ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:55.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a30ff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10c300 con 0x7fe19c101f30 2024-03-18T23:00:55.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a0e9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:55.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a0e9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58570/0 (socket says 172.21.15.132:58570) 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.165+0000 7fe1a0e9b700 1 -- 172.21.15.132:0/2940371442 learned_addr learned my addr 172.21.15.132:0/2940371442 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe1a0e9b700 1 -- 172.21.15.132:0/2940371442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c19dab0 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe1a0e9b700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe190005850 tx=0x7fe190005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe190026070 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19000bbd0 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c19dda0 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe190003970 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.166+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c197810 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.167+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fe190029070 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.167+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c103410 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.167+0000 7fe199ffb700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 0x7fe18c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.168+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe19001e070 con 0x7fe19c101f30 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.168+0000 7fe19bfff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 0x7fe18c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:55.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.168+0000 7fe19bfff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 0x7fe18c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe188006fd0 tx=0x7fe188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:55.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.173+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe190013070 con 0x7fe19c101f30 2024-03-18T23:00:55.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:55.376+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe19c0008d0 con 0x7fe19c101f30 2024-03-18T23:00:56.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.018+0000 7fe199ffb700 1 -- 172.21.15.132:0/2940371442 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe190030020 con 0x7fe19c101f30 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 msgr2=0x7fe18c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 0x7fe18c03b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe188006fd0 tx=0x7fe188006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 msgr2=0x7fe19c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe190005850 tx=0x7fe190005030 comp rx=0 tx=0).stop 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 shutdown_connections 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe18c038fc0 0x7fe18c03b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 --2- 172.21.15.132:0/2940371442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe19c101f30 0x7fe19c19d570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 >> 172.21.15.132:0/2940371442 conn(0x7fe19c0fd7e0 msgr2=0x7fe19c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 shutdown_connections 2024-03-18T23:00:56.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:00:56.021+0000 7fe1a30ff700 1 -- 172.21.15.132:0/2940371442 wait complete. 2024-03-18T23:00:56.023 INFO:teuthology.orchestra.run.smithi132.stderr:adjusted tunables profile to default 2024-03-18T23:00:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:56 smithi132 ceph-mon[32020]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:56.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:56 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T23:00:56.740 INFO:tasks.cephadm:Adding mon.a on smithi132 2024-03-18T23:00:56.740 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-03-18T23:00:56.740 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-03-18T23:00:56.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c' 2024-03-18T23:00:56.972 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:57.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T23:00:57.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:00:57.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:00:57.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:58.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 -- 172.21.15.196:0/214165345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 msgr2=0x7fcb80102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:58.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 --2- 172.21.15.196:0/214165345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80102310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009910 tx=0x7fcb70009c20 comp rx=0 tx=0).stop 2024-03-18T23:00:58.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 -- 172.21.15.196:0/214165345 shutdown_connections 2024-03-18T23:00:58.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 --2- 172.21.15.196:0/214165345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80102310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 -- 172.21.15.196:0/214165345 >> 172.21.15.196:0/214165345 conn(0x7fcb800fd7e0 msgr2=0x7fcb800ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 -- 172.21.15.196:0/214165345 shutdown_connections 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.337+0000 7fcb85be0700 1 -- 172.21.15.196:0/214165345 wait complete. 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb85be0700 1 Processor -- start 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb85be0700 1 -- start start 2024-03-18T23:00:58.342 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb85be0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb85be0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010c300 con 0x7fcb80101f30 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb7f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb7f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.196:55776/0 (socket says 172.21.15.196:55776) 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.338+0000 7fcb7f7fe700 1 -- 172.21.15.196:0/3674962037 learned_addr learned my addr 172.21.15.196:0/3674962037 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb7f7fe700 1 -- 172.21.15.196:0/3674962037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb80076500 con 0x7fcb80101f30 2024-03-18T23:00:58.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb7f7fe700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb70005850 tx=0x7fcb70005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb70026070 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb80074bb0 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb7000bc30 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb70003a20 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.339+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb80075070 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.340+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fcb70029070 con 0x7fcb80101f30 2024-03-18T23:00:58.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.340+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb80103410 con 0x7fcb80101f30 2024-03-18T23:00:58.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.340+0000 7fcb7cff9700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 0x7fcb6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:00:58.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.340+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb7001e070 con 0x7fcb80101f30 2024-03-18T23:00:58.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.340+0000 7fcb7effd700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 0x7fcb6803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:00:58.345 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.341+0000 7fcb7effd700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 0x7fcb6803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006fd0 tx=0x7fcb74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:00:58.346 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.345+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcb70013070 con 0x7fcb80101f30 2024-03-18T23:00:58.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:58 smithi132 ceph-mon[32020]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:00:58.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:58 smithi132 ceph-mon[32020]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:00:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:58 smithi132 ceph-mon[32020]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:00:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:58 smithi132 ceph-mon[32020]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:00:58.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.554+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fcb80075300 con 0x7fcb68038fc0 2024-03-18T23:00:58.560 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.560+0000 7fcb7cff9700 1 -- 172.21.15.196:0/3674962037 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcb80075300 con 0x7fcb68038fc0 2024-03-18T23:00:58.560 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-03-18T23:00:58.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 msgr2=0x7fcb6803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 0x7fcb6803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006fd0 tx=0x7fcb74006e40 comp rx=0 tx=0).stop 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 msgr2=0x7fcb80074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb70005850 tx=0x7fcb70005030 comp rx=0 tx=0).stop 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 shutdown_connections 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcb68038fc0 0x7fcb6803b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 --2- 172.21.15.196:0/3674962037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb80101f30 0x7fcb80074610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 >> 172.21.15.196:0/3674962037 conn(0x7fcb800fd7e0 msgr2=0x7fcb80103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 shutdown_connections 2024-03-18T23:00:58.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:00:58.563+0000 7fcb85be0700 1 -- 172.21.15.196:0/3674962037 wait complete. 2024-03-18T23:00:59.139 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.b.service 2024-03-18T23:00:59.142 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.c.service 2024-03-18T23:00:59.144 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T23:00:59.144 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T23:00:59.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:00:59.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: Saving service mon spec with placement smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c;count:3 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: Deploying daemon mon.c on smithi196 2024-03-18T23:00:59.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:00:59 smithi132 ceph-mon[32020]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:00.248 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 -- 172.21.15.196:0/3661285547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 msgr2=0x7f549c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/3661285547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c101230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5490009910 tx=0x7f5490009c20 comp rx=0 tx=0).stop 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 -- 172.21.15.196:0/3661285547 shutdown_connections 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/3661285547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c101230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 -- 172.21.15.196:0/3661285547 >> 172.21.15.196:0/3661285547 conn(0x7f549c0fc720 msgr2=0x7f549c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:01.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 -- 172.21.15.196:0/3661285547 shutdown_connections 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.963+0000 7f54a2e3c700 1 -- 172.21.15.196:0/3661285547 wait complete. 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a2e3c700 1 Processor -- start 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a2e3c700 1 -- start start 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a2e3c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a2e3c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f549c10b220 con 0x7f549c100e50 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a0bd8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a0bd8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.196:55792/0 (socket says 172.21.15.196:55792) 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.964+0000 7f54a0bd8700 1 -- 172.21.15.196:0/2361622589 learned_addr learned my addr 172.21.15.196:0/2361622589 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:01:01.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f54a0bd8700 1 -- 172.21.15.196:0/2361622589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c198b50 con 0x7f549c100e50 2024-03-18T23:01:01.967 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f54a0bd8700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5490005850 tx=0x7f5490005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5490026070 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c198e40 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549000bbd0 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5490003970 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.965+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c199300 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.966+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f5490029070 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.966+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c102330 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.966+0000 7f5499ffb700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.966+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f549001e070 con 0x7f549c100e50 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.966+0000 7f549bfff700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:01.968 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.967+0000 7f549bfff700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 0x7f548c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5488006fd0 tx=0x7f5488006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:01.972 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:01.972+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5490013070 con 0x7f549c100e50 2024-03-18T23:01:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.268+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f549c06a6f0 con 0x7f549c100e50 2024-03-18T23:01:02.269 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.268+0000 7f5499ffb700 1 -- 172.21.15.196:0/2361622589 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f5490030090 con 0x7f549c100e50 2024-03-18T23:01:02.269 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T23:01:02.269 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","modified":"2024-03-18T22:59:16.031249Z","created":"2024-03-18T22:59:16.031249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T23:01:02.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 msgr2=0x7f548c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:02.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 0x7f548c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5488006fd0 tx=0x7f5488006e40 comp rx=0 tx=0).stop 2024-03-18T23:01:02.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 msgr2=0x7f549c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5490005850 tx=0x7f5490005030 comp rx=0 tx=0).stop 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 shutdown_connections 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f548c038fc0 0x7f548c03b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 --2- 172.21.15.196:0/2361622589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c100e50 0x7f549c198610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 >> 172.21.15.196:0/2361622589 conn(0x7f549c0fc720 msgr2=0x7f549c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 shutdown_connections 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:02.271+0000 7f54a2e3c700 1 -- 172.21.15.196:0/2361622589 wait complete. 2024-03-18T23:01:02.272 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-18T23:01:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:02 smithi132 ceph-mon[32020]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:03.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:03 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/2361622589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:03.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:03 smithi196 podman[67070]: 2024-03-18 23:01:03.822236381 +0000 UTC m=+0.017775800 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:01:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:04 smithi132 ceph-mon[32020]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.406 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T23:01:04.406 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 podman[67070]: 2024-03-18 23:01:04.107964067 +0000 UTC m=+0.303503469 container init 54ceaf63f9e536b6d80e81936e6aa7836252478fa117404025e47f130d055d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 podman[67070]: 2024-03-18 23:01:04.111329275 +0000 UTC m=+0.306868683 container start 54ceaf63f9e536b6d80e81936e6aa7836252478fa117404025e47f130d055d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pidfile_write: ignore empty --pid-file 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: load: jerasure load: lrc 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: RocksDB version: 7.9.2 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Git sha 0 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: DB SUMMARY 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: DB Session ID: UNHN950QMN378F89S4RJ 2024-03-18T23:01:04.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: CURRENT file: CURRENT 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.error_if_exists: 0 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.create_if_missing: 0 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.paranoid_checks: 1 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.env: 0x55a6e73fbc60 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.info_log: 0x55a6e86e1ae0 2024-03-18T23:01:04.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.statistics: (nil) 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.use_fsync: 0 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_log_file_size: 0 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_fallocate: 1 2024-03-18T23:01:04.436 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.use_direct_reads: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.db_log_dir: 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.wal_dir: 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T23:01:04.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.write_buffer_manager: 0x55a6e86b41e0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.unordered_write: 0 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T23:01:04.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.row_cache: None 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.wal_filter: None 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.two_write_queues: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.wal_compression: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.atomic_flush: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.log_readahead_size: 0 2024-03-18T23:01:04.439 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_background_jobs: 2 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_background_compactions: -1 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_subcompactions: 1 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T23:01:04.440 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_open_files: -1 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_background_flushes: -1 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Compression algorithms supported: 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kZSTD supported: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kXpressCompression supported: 0 2024-03-18T23:01:04.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kZlibCompression supported: 1 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kSnappyCompression supported: 1 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kLZ4Compression supported: 1 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: kBZip2Compression supported: 0 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.merge_operator: 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_filter: None 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T23:01:04.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a6e86e1be0) 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-03-18T23:01:04.443 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x55a6e86e7090 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-03-18T23:01:04.444 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-03-18T23:01:04.445 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression: NoCompression 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.num_levels: 7 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T23:01:04.446 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T23:01:04.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T23:01:04.448 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T23:01:04.449 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.table_properties_collectors: 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.inplace_update_support: 0 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.bloom_locality: 0 2024-03-18T23:01:04.450 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.max_successive_merges: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.ttl: 2592000 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enable_blob_files: false 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.min_blob_size: 0 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T23:01:04.451 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 48f8b775-8f23-4a55-bc83-9d5b255a398a 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802864141247, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802864141673, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802864, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "48f8b775-8f23-4a55-bc83-9d5b255a398a", "db_session_id": "UNHN950QMN378F89S4RJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802864141753, "job": 1, "event": "recovery_finished"} 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T23:01:04.452 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a6e87da000 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: DB pointer 0x55a6e87c4000 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-18T23:01:04.453 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.454 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x55a6e86e7090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-18T23:01:04.455 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(???) e0 preinit fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).mds e1 new map 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T23:01:04.456 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.457 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:01:04.458 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Deploying cephadm binary to smithi196 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Added host smithi196 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.459 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:04.460 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Saving service mon spec with placement smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c;count:3 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:04.461 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: Deploying daemon mon.c on smithi196 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/2361622589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 ceph-mon[67089]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 bash[67070]: 54ceaf63f9e536b6d80e81936e6aa7836252478fa117404025e47f130d055d62 2024-03-18T23:01:04.462 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:04 smithi196 systemd[1]: Started Ceph mon.c for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:01:04.641 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 -- 172.21.15.196:0/2192694116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be0100f80 msgr2=0x7f3bc8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 --2- 172.21.15.196:0/2192694116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be0100f80 0x7f3bc8005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005fa0 tx=0x7f3bd000f4a0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 -- 172.21.15.196:0/2192694116 shutdown_connections 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 --2- 172.21.15.196:0/2192694116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be0100f80 0x7f3bc8005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 -- 172.21.15.196:0/2192694116 >> 172.21.15.196:0/2192694116 conn(0x7f3be00fc830 msgr2=0x7f3be00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 -- 172.21.15.196:0/2192694116 shutdown_connections 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.277+0000 7f3be54e4700 1 -- 172.21.15.196:0/2192694116 wait complete. 2024-03-18T23:01:09.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.278+0000 7f3be54e4700 1 Processor -- start 2024-03-18T23:01:09.280 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.278+0000 7f3be54e4700 1 -- start start 2024-03-18T23:01:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3be54e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:09.281 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3be54e4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 0x7f3be01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3be54e4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0108db0 con 0x7f3be019db60 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3be54e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0108c30 con 0x7f3be0100f80 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54116/0 (socket says 172.21.15.196:54116) 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 -- 172.21.15.196:0/3711963273 learned_addr learned my addr 172.21.15.196:0/3711963273 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bde7fc700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 0x7f3be01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 -- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 msgr2=0x7f3be019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 -- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 msgr2=0x7f3be019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bdeffd700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:01:09.282 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bde7fc700 1 -- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 msgr2=0x7f3be019d620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bde7fc700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bde7fc700 1 -- 172.21.15.196:0/3711963273 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be019e0e0 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.279+0000 7f3bde7fc700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 0x7f3be01976f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b4d0 tx=0x7f3bd400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.280+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3bd4011630 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.280+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4011cb0 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.280+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3bd4010e60 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.280+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be0197c90 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.280+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be0198220 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.281+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f3bd4018020 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.281+0000 7f3bc7fff700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 0x7f3bcc03b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.281+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0102460 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.281+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3bd404ba90 con 0x7f3be019db60 2024-03-18T23:01:09.283 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.281+0000 7f3bdeffd700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 0x7f3bcc03b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:09.284 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.282+0000 7f3bdeffd700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 0x7f3bcc03b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005fd0 tx=0x7f3bd0013a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:09.287 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.287+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3bd401f020 con 0x7f3be019db60 2024-03-18T23:01:09.561 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.561+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3be0198590 con 0x7f3be019db60 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: mon.a calling monitor election 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.562 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: mon.c calling monitor election 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: monmap epoch 2 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: last_changed 2024-03-18T23:01:04.164075+0000 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: min_mon_release 19 (squid) 2024-03-18T23:01:09.563 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: election_strategy: 1 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: fsmap 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: mgrmap e13: x(active, since 50s) 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: overall HEALTH_OK 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:09.564 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:09.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.561+0000 7f3bc7fff700 1 -- 172.21.15.196:0/3711963273 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f3bd4037090 con 0x7f3be019db60 2024-03-18T23:01:09.565 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T23:01:09.565 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","modified":"2024-03-18T23:01:04.164075Z","created":"2024-03-18T22:59:16.031249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 msgr2=0x7f3bcc03b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 0x7f3bcc03b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005fd0 tx=0x7f3bd0013a00 comp rx=0 tx=0).stop 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 msgr2=0x7f3be01976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 0x7f3be01976f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b4d0 tx=0x7f3bd400b890 comp rx=0 tx=0).stop 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 shutdown_connections 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3bcc038d60 0x7f3bcc03b220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3be0100f80 0x7f3be019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 --2- 172.21.15.196:0/3711963273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be019db60 0x7f3be01976f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 >> 172.21.15.196:0/3711963273 conn(0x7f3be00fc830 msgr2=0x7f3be00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:09.566 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 shutdown_connections 2024-03-18T23:01:09.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:09.564+0000 7f3be54e4700 1 -- 172.21.15.196:0/3711963273 wait complete. 2024-03-18T23:01:09.567 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: mon.a calling monitor election 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: mon.c calling monitor election 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: monmap epoch 2 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: last_changed 2024-03-18T23:01:04.164075+0000 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: min_mon_release 19 (squid) 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: election_strategy: 1 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:09.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: fsmap 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: mgrmap e13: x(active, since 50s) 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: overall HEALTH_OK 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:09.618 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: Deploying daemon mon.b on smithi144 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/3711963273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:10.469 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:10.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:10.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:10.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:10.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: Deploying daemon mon.b on smithi144 2024-03-18T23:01:10.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/3711963273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:10.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:11.089 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T23:01:11.089 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T23:01:11.334 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:11.445 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:11 smithi196 ceph-mon[67089]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:11.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:11 smithi132 ceph-mon[32020]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:12.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/3806175078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b8102070 msgr2=0x7ff9b8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:12.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/3806175078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b8102070 0x7ff9b8102450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009910 tx=0x7ff9a8009c20 comp rx=0 tx=0).stop 2024-03-18T23:01:12.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/3806175078 shutdown_connections 2024-03-18T23:01:12.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/3806175078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b8102070 0x7ff9b8102450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/3806175078 >> 172.21.15.196:0/3806175078 conn(0x7ff9b80fd8a0 msgr2=0x7ff9b80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/3806175078 shutdown_connections 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/3806175078 wait complete. 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.644+0000 7ff9bfa5f700 1 Processor -- start 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bfa5f700 1 -- start start 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bfa5f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bfa5f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 0x7ff9b81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bfa5f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b810c440 con 0x7ff9b819dc70 2024-03-18T23:01:12.646 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bfa5f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b810c2c0 con 0x7ff9b8102070 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:54128/0 (socket says 172.21.15.196:54128) 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 -- 172.21.15.196:0/1212085334 learned_addr learned my addr 172.21.15.196:0/1212085334 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 -- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 msgr2=0x7ff9b819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 -- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 msgr2=0x7ff9b819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T23:01:12.648 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bcffa700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 0x7ff9b81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bd7fb700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bcffa700 1 -- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 msgr2=0x7ff9b819d730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bcffa700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.645+0000 7ff9bcffa700 1 -- 172.21.15.196:0/1212085334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b819e2b0 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9bcffa700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 0x7ff9b81978f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b4d0 tx=0x7ff9b400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff9b4011630 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b8197e30 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b4011cb0 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b81983c0 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.646+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff9b4010e60 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.647+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b8198730 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.647+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7ff9b4018020 con 0x7ff9b819dc70 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.647+0000 7ff9ae7fc700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 0x7ff9a403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:12.649 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.647+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff9b404ba50 con 0x7ff9b819dc70 2024-03-18T23:01:12.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.647+0000 7ff9bd7fb700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 0x7ff9a403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:12.650 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.648+0000 7ff9bd7fb700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 0x7ff9a403b1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b5c0 tx=0x7ff9a8009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:12.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.652+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff9b401f020 con 0x7ff9b819dc70 2024-03-18T23:01:12.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.925+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff9b8198ec0 con 0x7ff9b819dc70 2024-03-18T23:01:12.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.926+0000 7ff9ae7fc700 1 -- 172.21.15.196:0/1212085334 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff9b400fe40 con 0x7ff9b819dc70 2024-03-18T23:01:12.927 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T23:01:12.927 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","modified":"2024-03-18T23:01:04.164075Z","created":"2024-03-18T22:59:16.031249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T23:01:12.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 msgr2=0x7ff9a403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 0x7ff9a403b1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b5c0 tx=0x7ff9a8009650 comp rx=0 tx=0).stop 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 msgr2=0x7ff9b81978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 0x7ff9b81978f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b4d0 tx=0x7ff9b400b890 comp rx=0 tx=0).stop 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 shutdown_connections 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff9a4038d00 0x7ff9a403b1c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8102070 0x7ff9b819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 --2- 172.21.15.196:0/1212085334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9b819dc70 0x7ff9b81978f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 >> 172.21.15.196:0/1212085334 conn(0x7ff9b80fd8a0 msgr2=0x7ff9b8103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 shutdown_connections 2024-03-18T23:01:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:12.928+0000 7ff9bfa5f700 1 -- 172.21.15.196:0/1212085334 wait complete. 2024-03-18T23:01:12.930 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_open_files: -1 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_background_flushes: -1 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Compression algorithms supported: 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T23:01:12.984 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kZSTD supported: 0 2024-03-18T23:01:13.213 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:12 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/1212085334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kXpressCompression supported: 0 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kZlibCompression supported: 1 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kSnappyCompression supported: 1 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kLZ4Compression supported: 1 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: kBZip2Compression supported: 0 2024-03-18T23:01:13.314 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.merge_operator: 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_filter: None 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5654959efbe0) 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T23:01:13.315 INFO:journalctl@ceph.mon.b.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_type: 0 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: data_block_index_type: 0 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_shortening: 1 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: checksum: 4 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: no_block_cache: 0 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache: 0x5654959f5090 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_options: 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: capacity : 536870912 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: num_shard_bits : 4 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: strict_capacity_limit : 0 2024-03-18T23:01:13.316 INFO:journalctl@ceph.mon.b.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_cache_compressed: (nil) 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: persistent_cache: (nil) 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_size: 4096 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_size_deviation: 10 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_restart_interval: 16 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: index_block_restart_interval: 1 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: metadata_block_size: 4096 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: partition_filters: 0 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: use_delta_encoding: 1 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: filter_policy: bloomfilter 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: whole_key_filtering: 1 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: verify_compression: 0 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: format_version: 5 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: enable_index_compression: 1 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: block_align: 0 2024-03-18T23:01:13.317 INFO:journalctl@ceph.mon.b.smithi144.stdout: max_auto_readahead_size: 262144 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout: prepopulate_block_cache: 0 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout: initial_auto_readahead_size: 8192 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression: NoCompression 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.num_levels: 7 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T23:01:13.318 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T23:01:13.319 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T23:01:13.320 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T23:01:13.321 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T23:01:13.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.table_properties_collectors: 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.inplace_update_support: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.bloom_locality: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.max_successive_merges: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T23:01:13.323 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.ttl: 2592000 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.enable_blob_files: false 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.min_blob_size: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T23:01:13.324 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3b403503-4845-4399-aa5c-b8a08cd83106 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802872982177, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802872982604, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710802872, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3b403503-4845-4399-aa5c-b8a08cd83106", "db_session_id": "BGP5M40WRCAMUW83OCV4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710802872982683, "job": 1, "event": "recovery_finished"} 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565495ae8000 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: DB pointer 0x565495ad2000 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** DB Stats ** 2024-03-18T23:01:13.325 INFO:journalctl@ceph.mon.b.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.326 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T23:01:13.327 INFO:journalctl@ceph.mon.b.smithi144.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: Block cache BinnedLRUCache@0x5654959f5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: starting mon.b rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(???) e0 preinit fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).mds e1 new map 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: e1 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: legacy client fscid: -1 2024-03-18T23:01:13.328 INFO:journalctl@ceph.mon.b.smithi144.stdout: 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout: No filesystems configured 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.329 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:12 smithi144 ceph-mon[66781]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.330 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Deploying cephadm binary to smithi196 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Added host smithi196 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2940371442' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T23:01:13.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:13.332 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Saving service mon spec with placement smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c;count:3 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Deploying daemon mon.c on smithi196 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.333 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/2361622589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: mon.a calling monitor election 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: mon.c calling monitor election 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.334 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: monmap epoch 2 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: last_changed 2024-03-18T23:01:04.164075+0000 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: min_mon_release 19 (squid) 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: election_strategy: 1 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: fsmap 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: mgrmap e13: x(active, since 50s) 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: overall HEALTH_OK 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:13.335 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: Deploying daemon mon.b on smithi144 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/3711963273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/1212085334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 ceph-mon[66781]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 bash[66759]: b5996aff448e7e1c1d10c3d43d86ed760599ee6bcfece39b35e7271c5b834be3 2024-03-18T23:01:13.336 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:13 smithi144 systemd[1]: Started Ceph mon.b for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:01:13.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:12 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/1212085334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T23:01:14.467 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T23:01:14.467 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T23:01:14.709 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: mon.c calling monitor election 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: mon.a calling monitor election 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: monmap epoch 3 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: min_mon_release 19 (squid) 2024-03-18T23:01:18.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: election_strategy: 1 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: fsmap 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: mgrmap e13: x(active, since 59s) 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: mon.c calling monitor election 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: mon.a calling monitor election 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:18.441 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: monmap epoch 3 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: min_mon_release 19 (squid) 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: election_strategy: 1 2024-03-18T23:01:18.442 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: fsmap 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: mgrmap e13: x(active, since 59s) 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:18.443 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:19.196 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.195+0000 7fe23e544700 1 -- 172.21.15.196:0/4216266884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2200035c0 msgr2=0x7fe220005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 --2- 172.21.15.196:0/4216266884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2200035c0 0x7fe220005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe238101d60 tx=0x7fe22800feb0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 -- 172.21.15.196:0/4216266884 shutdown_connections 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 --2- 172.21.15.196:0/4216266884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2200035c0 0x7fe220005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 -- 172.21.15.196:0/4216266884 >> 172.21.15.196:0/4216266884 conn(0x7fe2380fd920 msgr2=0x7fe2380ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 -- 172.21.15.196:0/4216266884 shutdown_connections 2024-03-18T23:01:19.197 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 -- 172.21.15.196:0/4216266884 wait complete. 2024-03-18T23:01:19.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 Processor -- start 2024-03-18T23:01:19.199 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.196+0000 7fe23e544700 1 -- start start 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe238197df0 0x7fe2381982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe238109ec0 con 0x7fe238197df0 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe238109d40 con 0x7fe2200035c0 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23e544700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810a040 con 0x7fe23819dc30 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:60982/0 (socket says 172.21.15.196:60982) 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 -- 172.21.15.196:0/119897306 learned_addr learned my addr 172.21.15.196:0/119897306 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe23cae1700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe238197df0 0x7fe2381982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe237fff700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:19.200 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe237fff700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 msgr2=0x7fe23819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe237fff700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 msgr2=0x7fe23819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe237fff700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe237fff700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 msgr2=0x7fe23819d6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe238197df0 msgr2=0x7fe2381982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe238197df0 0x7fe2381982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 -- 172.21.15.196:0/119897306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2381a45c0 con 0x7fe23819dc30 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.197+0000 7fe2377fe700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b780 tx=0x7fe22c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.198+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c004230 con 0x7fe23819dc30 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.198+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe22c0043d0 con 0x7fe23819dc30 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.198+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2381a47f0 con 0x7fe23819dc30 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.198+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c011c90 con 0x7fe23819dc30 2024-03-18T23:01:19.201 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.198+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2381a4d30 con 0x7fe23819dc30 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fe22c01a400 con 0x7fe23819dc30 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe238063940 con 0x7fe23819dc30 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe2357fa700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 0x7fe22403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe22c04bb30 con 0x7fe23819dc30 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe237fff700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 0x7fe22403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:19.202 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.199+0000 7fe237fff700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 0x7fe22403b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe22800f8d0 tx=0x7fe228005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:19.204 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.204+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe22c020500 con 0x7fe23819dc30 2024-03-18T23:01:19.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:19.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:19.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:19.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.495+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe2381a5020 con 0x7fe23819dc30 2024-03-18T23:01:19.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.496+0000 7fe2357fa700 1 -- 172.21.15.196:0/119897306 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fe22c0202a0 con 0x7fe23819dc30 2024-03-18T23:01:19.497 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-18T23:01:19.497 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","modified":"2024-03-18T23:01:13.004593Z","created":"2024-03-18T22:59:16.031249Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T23:01:19.498 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 msgr2=0x7fe22403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 0x7fe22403b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe22800f8d0 tx=0x7fe228005ad0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 msgr2=0x7fe2381978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b780 tx=0x7fe22c00ba90 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 shutdown_connections 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe224039450 0x7fe22403b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2200035c0 0x7fe23819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe238197df0 0x7fe2381982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 --2- 172.21.15.196:0/119897306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe23819dc30 0x7fe2381978b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 >> 172.21.15.196:0/119897306 conn(0x7fe2380fd920 msgr2=0x7fe2380fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 shutdown_connections 2024-03-18T23:01:19.499 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:19.498+0000 7fe23e544700 1 -- 172.21.15.196:0/119897306 wait complete. 2024-03-18T23:01:19.500 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-03-18T23:01:20.197 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-18T23:01:20.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-18T23:01:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: mon.c calling monitor election 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: mon.a calling monitor election 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: monmap epoch 3 2024-03-18T23:01:20.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: min_mon_release 19 (squid) 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: election_strategy: 1 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: fsmap 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: mgrmap e13: x(active, since 59s) 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T23:01:20.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:20.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:20.441 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:01:21.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mon.b calling monitor election 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mon.b calling monitor election 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mon.c calling monitor election 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mon.a calling monitor election 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: monmap epoch 3 2024-03-18T23:01:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: min_mon_release 19 (squid) 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: election_strategy: 1 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: fsmap 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: mgrmap e13: x(active, since 61s) 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Cluster is now healthy 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: overall HEALTH_OK 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:21.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:21 smithi144 ceph-mon[66781]: Reconfiguring daemon mon.a on smithi132 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mon.b calling monitor election 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mon.b calling monitor election 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mon.c calling monitor election 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mon.a calling monitor election 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:01:21.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: monmap epoch 3 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: min_mon_release 19 (squid) 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: election_strategy: 1 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: fsmap 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: mgrmap e13: x(active, since 61s) 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Cluster is now healthy 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: overall HEALTH_OK 2024-03-18T23:01:21.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:21.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:21 smithi132 ceph-mon[32020]: Reconfiguring daemon mon.a on smithi132 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mon.b calling monitor election 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mon.b calling monitor election 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mon.c calling monitor election 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mon.a calling monitor election 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: monmap epoch 3 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: last_changed 2024-03-18T23:01:13.004593+0000 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: created 2024-03-18T22:59:16.031249+0000 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: min_mon_release 19 (squid) 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: election_strategy: 1 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: 0: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.a 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: fsmap 2024-03-18T23:01:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: mgrmap e13: x(active, since 61s) 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Cluster is now healthy 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: overall HEALTH_OK 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:21.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:21 smithi196 ceph-mon[67089]: Reconfiguring daemon mon.a on smithi132 2024-03-18T23:01:22.256 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:22 smithi132 ceph-mon[32020]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:22.256 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:22 smithi144 ceph-mon[66781]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:22.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 -- 172.21.15.132:0/1749363281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828100e70 msgr2=0x7f5828101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:22.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 --2- 172.21.15.132:0/1749363281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828100e70 0x7f5828101250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f582000b0a0 tx=0x7f582000b3b0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 -- 172.21.15.132:0/1749363281 shutdown_connections 2024-03-18T23:01:22.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 --2- 172.21.15.132:0/1749363281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828100e70 0x7f5828101250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 -- 172.21.15.132:0/1749363281 >> 172.21.15.132:0/1749363281 conn(0x7f58280779b0 msgr2=0x7f58280fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:22.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 -- 172.21.15.132:0/1749363281 shutdown_connections 2024-03-18T23:01:22.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.472+0000 7f582cfee700 1 -- 172.21.15.132:0/1749363281 wait complete. 2024-03-18T23:01:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:22 smithi196 ceph-mon[67089]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:22.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 Processor -- start 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 -- start start 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5828100e70 0x7f582819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 0x7f5828198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f582810b1b0 con 0x7f582819dc00 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f582810b030 con 0x7f5828197dc0 2024-03-18T23:01:22.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582cfee700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f582810b330 con 0x7f5828100e70 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f5825d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f5825d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59230/0 (socket says 172.21.15.132:59230) 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f5825d9b700 1 -- 172.21.15.132:0/1826126875 learned_addr learned my addr 172.21.15.132:0/1826126875 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.473+0000 7f582659c700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5828100e70 0x7f582819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5826d9d700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 0x7f5828198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 -- 172.21.15.132:0/1826126875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5828100e70 msgr2=0x7f582819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5828100e70 0x7f582819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 -- 172.21.15.132:0/1826126875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 msgr2=0x7f5828198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 0x7f5828198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 -- 172.21.15.132:0/1826126875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58281a4590 con 0x7f582819dc00 2024-03-18T23:01:22.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5826d9d700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 0x7f5828198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f5825d9b700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b780 tx=0x7f581c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c004230 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581c0043d0 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58281a47c0 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c011c90 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.474+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58281a4cd0 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.475+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f581c01a400 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.475+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5828102300 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.475+0000 7f58177fe700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 0x7f581003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.475+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f581c04bb50 con 0x7f582819dc00 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.475+0000 7f582659c700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 0x7f581003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:22.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.476+0000 7f582659c700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 0x7f581003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58200062a0 tx=0x7f5820005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:22.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.480+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f581c020500 con 0x7f582819dc00 2024-03-18T23:01:22.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.678+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f58281a4ff0 con 0x7f582819dc00 2024-03-18T23:01:22.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.678+0000 7f58177fe700 1 -- 172.21.15.132:0/1826126875 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f581c0202a0 con 0x7f582819dc00 2024-03-18T23:01:22.680 INFO:teuthology.orchestra.run.smithi132.stdout:# minimal ceph.conf for 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:22.680 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-03-18T23:01:22.680 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:01:22.680 INFO:teuthology.orchestra.run.smithi132.stdout: mon_host = [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 msgr2=0x7f581003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 0x7f581003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58200062a0 tx=0x7f5820005040 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 msgr2=0x7f5828197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b780 tx=0x7f581c00ba90 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 shutdown_connections 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5810039450 0x7f581003b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5828197dc0 0x7f5828198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5828100e70 0x7f582819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 --2- 172.21.15.132:0/1826126875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f582819dc00 0x7f5828197880 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 >> 172.21.15.132:0/1826126875 conn(0x7f58280779b0 msgr2=0x7f58280ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:22.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 shutdown_connections 2024-03-18T23:01:22.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:22.681+0000 7f582cfee700 1 -- 172.21.15.132:0/1826126875 wait complete. 2024-03-18T23:01:23.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1826126875' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:23.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:23.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1826126875' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.386 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-18T23:01:23.387 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:01:23.387 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:01:23.416 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:01:23.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1826126875' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:23.483 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T23:01:23.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:01:23.513 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T23:01:23.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:23.580 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T23:01:23.580 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T23:01:23.615 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T23:01:23.615 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:01:23.685 INFO:tasks.cephadm:Adding mgr.x on smithi132 2024-03-18T23:01:23.685 INFO:tasks.cephadm:Adding mgr.y on smithi144 2024-03-18T23:01:23.685 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-03-18T23:01:23.685 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi132=x;smithi144=y;smithi196=z' 2024-03-18T23:01:23.956 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:01:24.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:24 smithi144 ceph-mon[66781]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:01:24.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:24 smithi144 ceph-mon[66781]: Reconfiguring daemon mon.b on smithi144 2024-03-18T23:01:24.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:24 smithi144 ceph-mon[66781]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:24.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:24 smithi132 ceph-mon[32020]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:01:24.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:24 smithi132 ceph-mon[32020]: Reconfiguring daemon mon.b on smithi144 2024-03-18T23:01:24.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:24 smithi132 ceph-mon[32020]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:24.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:24 smithi196 ceph-mon[67089]: Reconfiguring mon.b (monmap changed)... 2024-03-18T23:01:24.292 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:24 smithi196 ceph-mon[67089]: Reconfiguring daemon mon.b on smithi144 2024-03-18T23:01:24.292 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:24 smithi196 ceph-mon[67089]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: Reconfiguring daemon mon.c on smithi196 2024-03-18T23:01:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:25 smithi196 ceph-mon[67089]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:25.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: Reconfiguring daemon mon.c on smithi196 2024-03-18T23:01:25.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:25 smithi144 ceph-mon[66781]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: Reconfiguring mon.c (monmap changed)... 2024-03-18T23:01:25.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T23:01:25.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T23:01:25.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:25.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: Reconfiguring daemon mon.c on smithi196 2024-03-18T23:01:25.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:25 smithi132 ceph-mon[32020]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:26.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- 172.21.15.196:0/701338569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d01034b0 msgr2=0x7f46d01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:26.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 --2- 172.21.15.196:0/701338569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d01034b0 0x7f46d01038d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009910 tx=0x7f46c0009c20 comp rx=0 tx=0).stop 2024-03-18T23:01:26.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- 172.21.15.196:0/701338569 shutdown_connections 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 --2- 172.21.15.196:0/701338569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d01034b0 0x7f46d01038d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- 172.21.15.196:0/701338569 >> 172.21.15.196:0/701338569 conn(0x7f46d00fd410 msgr2=0x7f46d00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- 172.21.15.196:0/701338569 shutdown_connections 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- 172.21.15.196:0/701338569 wait complete. 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 Processor -- start 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.287+0000 7f46d63ac700 1 -- start start 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 0x7f46d01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 0x7f46d019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0107f20 con 0x7f46d0197d00 2024-03-18T23:01:26.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0107da0 con 0x7f46d01034b0 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d63ac700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01080a0 con 0x7f46d0198180 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d4949700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 0x7f46d019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d4949700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:37696/0 (socket says 172.21.15.196:37696) 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d4949700 1 -- 172.21.15.196:0/1281190125 learned_addr learned my addr 172.21.15.196:0/1281190125 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cffff700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 0x7f46d01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 -- 172.21.15.196:0/1281190125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 msgr2=0x7f46d019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 -- 172.21.15.196:0/1281190125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 msgr2=0x7f46d01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 0x7f46d01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 -- 172.21.15.196:0/1281190125 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d019fb80 con 0x7f46d0197d00 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46d4949700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cffff700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 0x7f46d01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:01:26.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.288+0000 7f46cf7fe700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 0x7f46d019cd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b550 tx=0x7f46c400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.289+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4004230 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.289+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c40043d0 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.289+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d019fe70 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.289+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4011c20 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.289+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01a03a0 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f46c40036a0 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d0063940 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46cd7fa700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 0x7f46b803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f46c404bb50 con 0x7f46d0197d00 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46cffff700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 0x7f46b803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:26.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.290+0000 7f46cffff700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 0x7f46b803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005330 tx=0x7f46c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:26.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.299+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f46c4021e00 con 0x7f46d0197d00 2024-03-18T23:01:26.505 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.504+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi144=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f46d00628a0 con 0x7f46b8039450 2024-03-18T23:01:26.511 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.511+0000 7f46cd7fa700 1 -- 172.21.15.196:0/1281190125 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f46d00628a0 con 0x7f46b8039450 2024-03-18T23:01:26.511 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 msgr2=0x7f46b803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 0x7f46b803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005330 tx=0x7f46c0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 msgr2=0x7f46d019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 0x7f46d019cd20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b550 tx=0x7f46c400b860 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 shutdown_connections 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b8039450 0x7f46b803b910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46d01034b0 0x7f46d01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46d0198180 0x7f46d019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 --2- 172.21.15.196:0/1281190125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46d0197d00 0x7f46d019cd20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 >> 172.21.15.196:0/1281190125 conn(0x7f46d00fd410 msgr2=0x7f46d00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 shutdown_connections 2024-03-18T23:01:26.514 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:01:26.513+0000 7f46d63ac700 1 -- 172.21.15.196:0/1281190125 wait complete. 2024-03-18T23:01:27.061 DEBUG:teuthology.orchestra.run.smithi144:mgr.y> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.y.service 2024-03-18T23:01:27.064 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.z.service 2024-03-18T23:01:27.066 INFO:tasks.cephadm:Deploying OSDs... 2024-03-18T23:01:27.066 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:01:27.066 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:01:27.083 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:01:27.083 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-18T23:01:27.140 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:27.141 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:27.141 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-18 23:00:31.038603636 +0000 2024-03-18T23:01:27.141 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-18 22:29:59.627319463 +0000 2024-03-18T23:01:27.141 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-18 22:29:59.627319463 +0000 2024-03-18T23:01:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-18 22:29:59.627319463 +0000 2024-03-18T23:01:27.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:01:27.207 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-18T23:01:27.207 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-18T23:01:27.207 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000197761 s, 2.6 MB/s 2024-03-18T23:01:27.209 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:01:27.265 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-03-18T23:01:27.320 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:01:27.320 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:27.320 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-18T23:01:27.320 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:27.321 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:27.321 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-18 23:00:31.074602703 +0000 2024-03-18T23:01:27.321 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-18 22:29:59.970310500 +0000 2024-03-18T23:01:27.321 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-18 22:29:59.970310500 +0000 2024-03-18T23:01:27.321 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-18 22:29:59.970310500 +0000 2024-03-18T23:01:27.321 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:27.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:27 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:27.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:27 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:27.383 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:27.384 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:27 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:27.386 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-18T23:01:27.386 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-18T23:01:27.386 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000219916 s, 2.3 MB/s 2024-03-18T23:01:27.388 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:01:27.443 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-03-18T23:01:27.498 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:01:27.498 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:27.498 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-18T23:01:27.498 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:27.498 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:27.499 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-18 23:00:31.110601770 +0000 2024-03-18T23:01:27.499 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-18 22:30:00.310301615 +0000 2024-03-18T23:01:27.499 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-18 22:30:00.310301615 +0000 2024-03-18T23:01:27.499 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-18 22:30:00.310301615 +0000 2024-03-18T23:01:27.499 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:01:27.563 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-18T23:01:27.563 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-18T23:01:27.564 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000238681 s, 2.1 MB/s 2024-03-18T23:01:27.565 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:01:27.621 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-03-18T23:01:27.675 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-18 23:00:31.147600812 +0000 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-18 22:30:00.657292547 +0000 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-18 22:30:00.657292547 +0000 2024-03-18T23:01:27.676 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-18 22:30:00.657292547 +0000 2024-03-18T23:01:27.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:01:27.740 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-03-18T23:01:27.741 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-03-18T23:01:27.741 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000213554 s, 2.4 MB/s 2024-03-18T23:01:27.742 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:01:27.798 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-18T23:01:27.798 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:01:27.816 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:01:27.816 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-18 23:01:12.433756394 +0000 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-18 22:43:24.334167626 +0000 2024-03-18T23:01:27.874 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-18 22:43:24.334167626 +0000 2024-03-18T23:01:27.875 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-18 22:43:24.334167626 +0000 2024-03-18T23:01:27.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:01:27.946 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-18T23:01:27.946 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-18T23:01:27.946 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000295444 s, 1.7 MB/s 2024-03-18T23:01:27.948 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:01:28.006 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-18 23:01:12.434756377 +0000 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-18 22:43:24.674161873 +0000 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-18 22:43:24.674161873 +0000 2024-03-18T23:01:28.067 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-18 22:43:24.674161873 +0000 2024-03-18T23:01:28.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:01:28.136 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-18T23:01:28.136 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-18T23:01:28.137 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000217974 s, 2.3 MB/s 2024-03-18T23:01:28.137 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:01:28.194 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-18T23:01:28.202 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:28 smithi196 ceph-mon[67089]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi144=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:28.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:28 smithi196 ceph-mon[67089]: Saving service mgr spec with placement smithi132=x;smithi144=y;smithi196=z;count:3 2024-03-18T23:01:28.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:28 smithi196 ceph-mon[67089]: Deploying daemon mgr.z on smithi196 2024-03-18T23:01:28.203 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:28 smithi196 ceph-mon[67089]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:28.250 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-18 23:01:12.434756377 +0000 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-18 22:43:25.026155917 +0000 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-18 22:43:25.026155917 +0000 2024-03-18T23:01:28.251 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-18 22:43:25.026155917 +0000 2024-03-18T23:01:28.251 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:01:28.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:28 smithi144 ceph-mon[66781]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi144=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:28.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:28 smithi144 ceph-mon[66781]: Saving service mgr spec with placement smithi132=x;smithi144=y;smithi196=z;count:3 2024-03-18T23:01:28.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:28 smithi144 ceph-mon[66781]: Deploying daemon mgr.z on smithi196 2024-03-18T23:01:28.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:28 smithi144 ceph-mon[66781]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:28.317 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-18T23:01:28.318 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-18T23:01:28.318 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000261384 s, 2.0 MB/s 2024-03-18T23:01:28.319 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:01:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:28 smithi132 ceph-mon[32020]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi132=x;smithi144=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:28 smithi132 ceph-mon[32020]: Saving service mgr spec with placement smithi132=x;smithi144=y;smithi196=z;count:3 2024-03-18T23:01:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:28 smithi132 ceph-mon[32020]: Deploying daemon mgr.z on smithi196 2024-03-18T23:01:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:28 smithi132 ceph-mon[32020]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:28.376 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-18T23:01:28.431 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:01:28.431 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-18 23:01:12.434756377 +0000 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-18 22:43:25.359150283 +0000 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-18 22:43:25.359150283 +0000 2024-03-18T23:01:28.432 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-18 22:43:25.359150283 +0000 2024-03-18T23:01:28.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:01:28.497 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-18T23:01:28.498 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-18T23:01:28.498 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000197722 s, 2.6 MB/s 2024-03-18T23:01:28.499 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:01:28.554 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-18T23:01:28.554 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T23:01:28.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T23:01:28.570 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-03-18T23:01:28.625 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T23:01:28.625 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-18 23:01:28.498401076 +0000 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-18 22:49:55.424267224 +0000 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-18 22:49:55.424267224 +0000 2024-03-18T23:01:28.626 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-18 22:49:55.424267224 +0000 2024-03-18T23:01:28.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T23:01:28.695 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-18T23:01:28.695 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-18T23:01:28.695 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000188537 s, 2.7 MB/s 2024-03-18T23:01:28.696 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T23:01:28.752 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-18 23:01:28.498401076 +0000 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-18 22:49:55.769259994 +0000 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-18 22:49:55.769259994 +0000 2024-03-18T23:01:28.807 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-18 22:49:55.769259994 +0000 2024-03-18T23:01:28.807 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T23:01:28.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:28 smithi196 podman[68902]: 2024-03-18 23:01:28.798605448 +0000 UTC m=+0.133937607 container create 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-18T23:01:28.873 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-18T23:01:28.873 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-18T23:01:28.873 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000209924 s, 2.4 MB/s 2024-03-18T23:01:28.874 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T23:01:28.929 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-03-18T23:01:28.995 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T23:01:28.995 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:28.995 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-18 23:01:28.498401076 +0000 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-18 22:49:56.107252910 +0000 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-18 22:49:56.107252910 +0000 2024-03-18T23:01:28.996 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-18 22:49:56.107252910 +0000 2024-03-18T23:01:28.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T23:01:29.061 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-18T23:01:29.061 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-18T23:01:29.061 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000190099 s, 2.7 MB/s 2024-03-18T23:01:29.062 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T23:01:29.117 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-03-18T23:01:29.172 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-18 23:01:28.499401055 +0000 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-18 22:49:56.434246057 +0000 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-18 22:49:56.434246057 +0000 2024-03-18T23:01:29.173 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-18 22:49:56.434246057 +0000 2024-03-18T23:01:29.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T23:01:29.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 podman[68902]: 2024-03-18 23:01:29.000999815 +0000 UTC m=+0.336331985 container init 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-18T23:01:29.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 podman[68902]: 2024-03-18 23:01:29.003505233 +0000 UTC m=+0.338837385 container start 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-18T23:01:29.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 ceph-mgr[69027]: -- 172.21.15.196:0/2370630288 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644906945a0 con 0x564490668c00 2024-03-18T23:01:29.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 bash[68902]: 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 2024-03-18T23:01:29.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 systemd[1]: Started Ceph mgr.z for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:01:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-18T23:01:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-18T23:01:29.238 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000206177 s, 2.5 MB/s 2024-03-18T23:01:29.239 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T23:01:29.295 INFO:tasks.cephadm:Deploying osd.0 on smithi132 with /dev/vg_nvme/lv_4... 2024-03-18T23:01:29.295 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:01:29.536 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:01:29.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:29.563+0000 7f1fef81f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:01:29.974 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:29.656+0000 7f1fef81f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:01:30.151 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:30.151 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:30.152 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.317 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:30.293+0000 7f1fef81f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:30.318 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:30.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:30.319 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:30.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:30.577 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:30.391+0000 7f1fef81f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:01:30.578 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:30.538+0000 7f1fef81f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:01:30.974 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:30.638+0000 7f1fef81f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:01:30.974 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:30.726+0000 7f1fef81f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:01:31.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:31 smithi144 ceph-mon[66781]: Deploying daemon mgr.y on smithi144 2024-03-18T23:01:31.298 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-mon[67089]: Deploying daemon mgr.y on smithi144 2024-03-18T23:01:31.298 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:31.001+0000 7f1fef81f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:01:31.298 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:31.113+0000 7f1fef81f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:01:31.299 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:31.200+0000 7f1fef81f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:01:31.299 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:31.297+0000 7f1fef81f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:01:31.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:31 smithi132 ceph-mon[32020]: Deploying daemon mgr.y on smithi144 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114519 s, 916 MB/s 2024-03-18T23:01:31.375 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-18T23:01:31.553 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:31.478+0000 7f1fef81f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:01:31.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:31 smithi144 systemd[1]: Started Ceph mgr.y for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:01:31.992 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2024-03-18T23:01:32.229 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:01:32.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:32.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.352 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:32.127+0000 7fa09d6d3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:01:32.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:32.224+0000 7fa09d6d3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:01:32.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:32.859 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:32.493+0000 7f1fef81f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:01:32.859 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:32.632+0000 7f1fef81f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:01:32.999 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:32.905+0000 7fa09d6d3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:01:33.000 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:32.998+0000 7fa09d6d3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:01:33.111 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:32.978+0000 7f1fef81f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:01:33.111 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:33.069+0000 7f1fef81f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:01:33.327 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.142+0000 7fa09d6d3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:01:33.327 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.243+0000 7fa09d6d3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:01:33.327 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.326+0000 7fa09d6d3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:01:33.462 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:33.267+0000 7f1fef81f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:01:33.462 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:33.461+0000 7f1fef81f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 -- 172.21.15.132:0/1424627458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4101240 msgr2=0x7fcaf4101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1424627458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4101240 0x7fcaf4101660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcae40099c0 tx=0x7fcae4009cd0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 -- 172.21.15.132:0/1424627458 shutdown_connections 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1424627458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 0x7fcaf4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1424627458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4101240 0x7fcaf4101660 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1424627458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4102440 0x7fcaf41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 -- 172.21.15.132:0/1424627458 >> 172.21.15.132:0/1424627458 conn(0x7fcaf40fc7e0 msgr2=0x7fcaf40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:33.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 -- 172.21.15.132:0/1424627458 shutdown_connections 2024-03-18T23:01:33.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 -- 172.21.15.132:0/1424627458 wait complete. 2024-03-18T23:01:33.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.666+0000 7fcafbc59700 1 Processor -- start 2024-03-18T23:01:33.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 -- start start 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4101240 0x7fcaf406b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 0x7fcaf419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410ce50 con 0x7fcaf4102440 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410ccd0 con 0x7fcaf4101240 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafbc59700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf410cfd0 con 0x7fcaf4102de0 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41664/0 (socket says 172.21.15.132:41664) 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 -- 172.21.15.132:0/1405261431 learned_addr learned my addr 172.21.15.132:0/1405261431 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf99f5700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4101240 0x7fcaf406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcafa1f6700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 0x7fcaf419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:33.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 -- 172.21.15.132:0/1405261431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 msgr2=0x7fcaf419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 0x7fcaf419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 -- 172.21.15.132:0/1405261431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4101240 msgr2=0x7fcaf406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4101240 0x7fcaf406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 -- 172.21.15.132:0/1405261431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf41a01f0 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.667+0000 7fcaf91f4700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d950 tx=0x7fcaf000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.668+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0016070 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.668+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf000fd70 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.668+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf41a04e0 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.668+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e610 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.668+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41a0a70 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fcaf0019070 con 0x7fcaf4102440 2024-03-18T23:01:33.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf4068fd0 con 0x7fcaf4102440 2024-03-18T23:01:33.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcaeaffd700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 0x7fcae003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:33.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcaf004bf20 con 0x7fcaf4102440 2024-03-18T23:01:33.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcaf99f5700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 0x7fcae003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:33.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.669+0000 7fcaf99f5700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 0x7fcae003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b580 tx=0x7fcae4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:33.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.674+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcaf0010e10 con 0x7fcaf4102440 2024-03-18T23:01:33.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.596+0000 7fa09d6d3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:01:33.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.705+0000 7fa09d6d3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:01:33.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.790+0000 7fa09d6d3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:01:33.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:33.888+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcaf4062750 con 0x7fcae0039200 2024-03-18T23:01:34.009 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:33.749+0000 7f1fef81f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:01:34.010 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:34.009+0000 7f1fef81f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:01:34.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:33.884+0000 7fa09d6d3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:01:34.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:34.059+0000 7fa09d6d3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:01:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:34.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:34.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.473 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:34.244+0000 7f1fef81f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:34.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:35 smithi132 ceph-mon[32020]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:01:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:35 smithi132 ceph-mon[32020]: Reconfiguring daemon mgr.x on smithi132 2024-03-18T23:01:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:35 smithi132 ceph-mon[32020]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:35.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-mon[66781]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:01:35.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-mon[66781]: Reconfiguring daemon mgr.x on smithi132 2024-03-18T23:01:35.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-mon[66781]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:35.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.021+0000 7fa09d6d3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:01:35.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.151+0000 7fa09d6d3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:01:35.473 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:35.037+0000 7f1fef81f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:01:35.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-mon[67089]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T23:01:35.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-mon[67089]: Reconfiguring daemon mgr.x on smithi132 2024-03-18T23:01:35.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-mon[67089]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:01:35.775 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.489+0000 7fa09d6d3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:01:35.775 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.579+0000 7fa09d6d3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:01:35.775 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.774+0000 7fa09d6d3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:01:35.922 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:35.613+0000 7f1fef81f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:01:35.923 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:35.709+0000 7f1fef81f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:01:35.923 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:01:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:01:35.922+0000 7f1fef81f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:01:36.079 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:35.961+0000 7fa09d6d3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: Standby manager daemon z started 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:01:36.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:36.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:01:36.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:36 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:36.290 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:36.289+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 14) ==== 91699+0+0 (secure 0 0 0) 0x7fcaf0016210 con 0x7fcaf4102440 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: Standby manager daemon z started 2024-03-18T23:01:36.292 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:01:36.292 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:36.292 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:01:36.292 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:36 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:36.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:36.245+0000 7fa09d6d3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:01:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: Standby manager daemon z started 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:01:36.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2128306416' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:36.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:36.505+0000 7fa09d6d3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:01:36.859 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:36.741+0000 7fa09d6d3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:01:37.609 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:37.540+0000 7fa09d6d3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:01:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:37 smithi144 ceph-mon[66781]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-18T23:01:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:01:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:37 smithi144 ceph-mon[66781]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:37.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:37 smithi132 ceph-mon[32020]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-18T23:01:37.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:01:37.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:37 smithi132 ceph-mon[32020]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:37 smithi196 ceph-mon[67089]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-18T23:01:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:01:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:37 smithi196 ceph-mon[67089]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:38.425 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:38.113+0000 7fa09d6d3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:01:38.425 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:38.209+0000 7fa09d6d3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:01:38.425 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:01:38.425+0000 7fa09d6d3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:01:38.426 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]: dispatch 2024-03-18T23:01:38.426 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]': finished 2024-03-18T23:01:38.426 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-mon[66781]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:01:38.426 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:38.426 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2363258022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:38.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]: dispatch 2024-03-18T23:01:38.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]': finished 2024-03-18T23:01:38.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:38 smithi132 ceph-mon[32020]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:01:38.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:38 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:38.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2363258022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:38.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]: dispatch 2024-03-18T23:01:38.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2013671741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8845c7f-34df-49a6-96ee-d71d153dc0f2"}]': finished 2024-03-18T23:01:38.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:38 smithi196 ceph-mon[67089]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T23:01:38.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:38 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:38.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2363258022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:01:39.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:39.315+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mon.0 v2:172.21.15.132:3300/0 8 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fcaf000a250 con 0x7fcaf4102440 2024-03-18T23:01:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: Standby manager daemon y started 2024-03-18T23:01:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:01:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:39.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:01:39.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:39.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:39 smithi144 ceph-mon[66781]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:39.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: Standby manager daemon y started 2024-03-18T23:01:39.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:01:39.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:39.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:01:39.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:39.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:39 smithi132 ceph-mon[32020]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: Standby manager daemon y started 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/3785348221' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:01:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:39 smithi196 ceph-mon[67089]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:40 smithi144 ceph-mon[66781]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-18T23:01:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:01:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:40 smithi132 ceph-mon[32020]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-18T23:01:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:01:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:40 smithi196 ceph-mon[67089]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-03-18T23:01:40.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:01:41.556 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:41 smithi132 ceph-mon[32020]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:41.603 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:41 smithi196 ceph-mon[67089]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:41 smithi144 ceph-mon[66781]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:44 smithi144 ceph-mon[66781]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:44 smithi132 ceph-mon[32020]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:44 smithi196 ceph-mon[67089]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:45.348 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:01:45.348 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:45.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:01:45.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:45.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T23:01:45.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:46.142 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:46 smithi132 ceph-mon[32020]: Deploying daemon osd.0 on smithi132 2024-03-18T23:01:46.142 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:46 smithi132 ceph-mon[32020]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:46 smithi144 ceph-mon[66781]: Deploying daemon osd.0 on smithi132 2024-03-18T23:01:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:46 smithi144 ceph-mon[66781]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:46.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:46 smithi196 ceph-mon[67089]: Deploying daemon osd.0 on smithi132 2024-03-18T23:01:46.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:46 smithi196 ceph-mon[67089]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:48.286 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:48 smithi132 ceph-mon[32020]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:48 smithi144 ceph-mon[66781]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:48.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:48 smithi196 ceph-mon[67089]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:50 smithi144 ceph-mon[66781]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:50 smithi132 ceph-mon[32020]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:50 smithi196 ceph-mon[67089]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:52.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:52 smithi132 ceph-mon[32020]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:52.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:52 smithi144 ceph-mon[66781]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:52.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:52 smithi196 ceph-mon[67089]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:53.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:53.505 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:53.505 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.505 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:53 smithi132 ceph-mon[32020]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:53.547 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 0 on host 'smithi132' 2024-03-18T23:01:53.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.545+0000 7fcaeaffd700 1 -- 172.21.15.132:0/1405261431 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcaf4062750 con 0x7fcae0039200 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 msgr2=0x7fcae003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 0x7fcae003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b580 tx=0x7fcae4005fb0 comp rx=0 tx=0).stop 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 msgr2=0x7fcaf419d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d950 tx=0x7fcaf000dc60 comp rx=0 tx=0).stop 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 shutdown_connections 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fcae0039200 0x7fcae003b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf4101240 0x7fcaf406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf4102de0 0x7fcaf419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:53.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 --2- 172.21.15.132:0/1405261431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaf4102440 0x7fcaf419d290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:53.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 >> 172.21.15.132:0/1405261431 conn(0x7fcaf40fc7e0 msgr2=0x7fcaf4105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:53.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 shutdown_connections 2024-03-18T23:01:53.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:01:53.548+0000 7fcafbc59700 1 -- 172.21.15.132:0/1405261431 wait complete. 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:53 smithi196 ceph-mon[67089]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:53 smithi144 ceph-mon[66781]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:54.141 DEBUG:teuthology.orchestra.run.smithi132:osd.0> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.0.service 2024-03-18T23:01:54.144 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_4... 2024-03-18T23:01:54.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:01:54.389 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:01:54.771 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:54 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.771 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:54 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.771 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:54 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:54.771 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:54 smithi144 ceph-mon[66781]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:01:54.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:54 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:54 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:54 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:54.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:54 smithi132 ceph-mon[32020]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:01:54.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:54 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:54 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:54.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:54 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:01:54.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:54 smithi196 ceph-mon[67089]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T23:01:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:55 smithi144 ceph-mon[66781]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:01:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:55 smithi144 ceph-mon[66781]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:01:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:55 smithi144 ceph-mon[66781]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:01:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:55 smithi144 ceph-mon[66781]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-mon[32020]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:01:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-mon[32020]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:01:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-mon[32020]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:01:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-mon[32020]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:55.866 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0[46793]: 2024-03-18T23:01:55.553+0000 7f564723d700 -1 osd.0 0 waiting for initial osdmap 2024-03-18T23:01:55.866 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:01:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0[46793]: 2024-03-18T23:01:55.556+0000 7f5643030700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:01:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:01:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:01:55.972 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-18T23:01:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-18T23:01:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013118 s, 799 MB/s 2024-03-18T23:01:55.973 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 2024-03-18T23:01:55.973 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-18T23:01:55.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:55 smithi196 ceph-mon[67089]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T23:01:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:55 smithi196 ceph-mon[67089]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T23:01:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:55 smithi196 ceph-mon[67089]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:01:55.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:55.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:55 smithi196 ceph-mon[67089]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:56 smithi144 ceph-mon[66781]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:01:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:56 smithi144 ceph-mon[66781]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:01:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:56.651 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-03-18T23:01:56.739 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:56 smithi132 ceph-mon[32020]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:01:56.739 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:56 smithi132 ceph-mon[32020]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:01:56.739 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:56.739 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:56.890 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:01:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:56 smithi196 ceph-mon[67089]: from='osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:01:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:56 smithi196 ceph-mon[67089]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T23:01:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:57.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:01:57.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:01:57.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] boot 2024-03-18T23:01:57.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:01:57.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi132 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 17088M 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:57 smithi144 ceph-mon[66781]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] boot 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi132 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 17088M 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:57.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:57 smithi132 ceph-mon[32020]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: osd.0 [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] boot 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi132 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 17088M 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:01:57.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:57 smithi196 ceph-mon[67089]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T23:01:58.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- 172.21.15.144:0/1331237133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 msgr2=0x7fb3b0109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:58.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 --2- 172.21.15.144:0/1331237133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b0109690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b0d0 tx=0x7fb3a800b3e0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- 172.21.15.144:0/1331237133 shutdown_connections 2024-03-18T23:01:58.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 --2- 172.21.15.144:0/1331237133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b0109690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 --2- 172.21.15.144:0/1331237133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0101240 0x7fb3b0101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 --2- 172.21.15.144:0/1331237133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0102440 0x7fb3b01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- 172.21.15.144:0/1331237133 >> 172.21.15.144:0/1331237133 conn(0x7fb3b00fc800 msgr2=0x7fb3b00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- 172.21.15.144:0/1331237133 shutdown_connections 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- 172.21.15.144:0/1331237133 wait complete. 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 Processor -- start 2024-03-18T23:01:58.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.332+0000 7fb3b582b700 1 -- start start 2024-03-18T23:01:58.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0101240 0x7fb3b0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 0x7fb3b0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010ce90 con 0x7fb3b0102440 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010cd10 con 0x7fb3b0102de0 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010d010 con 0x7fb3b0101240 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44970/0 (socket says 172.21.15.144:44970) 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 -- 172.21.15.144:0/2764341154 learned_addr learned my addr 172.21.15.144:0/2764341154 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3a67fc700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 0x7fb3b0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3aeffd700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0101240 0x7fb3b0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 -- 172.21.15.144:0/2764341154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0101240 msgr2=0x7fb3b0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0101240 0x7fb3b0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 -- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 msgr2=0x7fb3b0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 0x7fb3b0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 -- 172.21.15.144:0/2764341154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b01a01c0 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3a67fc700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 0x7fb3b0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3af7fe700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006070 tx=0x7fb3a8007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8021020 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b01a0450 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a801d950 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.333+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8005510 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.334+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01a09c0 con 0x7fb3b0102de0 2024-03-18T23:01:58.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.334+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b0068fd0 con 0x7fb3b0102de0 2024-03-18T23:01:58.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.335+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fb3a8024020 con 0x7fb3b0102de0 2024-03-18T23:01:58.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.336+0000 7fb3b4829700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 0x7fb39c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:01:58.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.336+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb3a80ce750 con 0x7fb3b0102de0 2024-03-18T23:01:58.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.336+0000 7fb3aeffd700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 0x7fb39c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:01:58.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.337+0000 7fb3aeffd700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 0x7fb39c0a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005950 tx=0x7fb3a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:01:58.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.340+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3a802b020 con 0x7fb3b0102de0 2024-03-18T23:01:58.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:01:58.543+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb3b00fe8f0 con 0x7fb39c0a2800 2024-03-18T23:01:59.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:58 smithi144 ceph-mon[66781]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:01:59.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:59.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:59.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:59.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:58 smithi132 ceph-mon[32020]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:01:59.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:59.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:59.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:01:59.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:58 smithi196 ceph-mon[67089]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T23:01:59.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:01:59.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:01:59.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:00.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:59 smithi144 ceph-mon[66781]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:00.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:01:59 smithi144 ceph-mon[66781]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:00.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:59 smithi132 ceph-mon[32020]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:00.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:01:59 smithi132 ceph-mon[32020]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:00.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:59 smithi196 ceph-mon[67089]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:01:59 smithi196 ceph-mon[67089]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/2083063388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:00 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:00 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]': finished 2024-03-18T23:02:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:00 smithi196 ceph-mon[67089]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:02:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:01.241 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/2083063388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.241 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:00 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.241 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:00 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]': finished 2024-03-18T23:02:01.242 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:00 smithi144 ceph-mon[66781]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:02:01.242 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:01.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/2083063388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:00 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]: dispatch 2024-03-18T23:02:01.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:00 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e2cb994-9178-41e5-9724-62360d566915"}]': finished 2024-03-18T23:02:01.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:00 smithi132 ceph-mon[32020]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T23:02:01.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:01 smithi196 ceph-mon[67089]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/1118593881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:01 smithi144 ceph-mon[66781]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/1118593881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:01 smithi132 ceph-mon[32020]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/1118593881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:04 smithi144 ceph-mon[66781]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:04 smithi132 ceph-mon[32020]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:04.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:04 smithi196 ceph-mon[67089]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:06.315 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:06 smithi144 ceph-mon[66781]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:06.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:06 smithi132 ceph-mon[32020]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:06 smithi196 ceph-mon[67089]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:08 smithi144 ceph-mon[66781]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:08.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:08 smithi132 ceph-mon[32020]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:08 smithi196 ceph-mon[67089]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:09.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:02:09.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:09.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:02:09.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T23:02:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:10.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:10 smithi144 ceph-mon[66781]: Deploying daemon osd.1 on smithi144 2024-03-18T23:02:10.120 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:10 smithi144 ceph-mon[66781]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:10.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:10 smithi132 ceph-mon[32020]: Deploying daemon osd.1 on smithi144 2024-03-18T23:02:10.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:10 smithi132 ceph-mon[32020]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:10.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:10 smithi196 ceph-mon[67089]: Deploying daemon osd.1 on smithi144 2024-03-18T23:02:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:10 smithi196 ceph-mon[67089]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:12.283 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:12 smithi144 ceph-mon[66781]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:12.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:12 smithi132 ceph-mon[32020]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:12.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:12 smithi196 ceph-mon[67089]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:14 smithi144 ceph-mon[66781]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:14 smithi132 ceph-mon[32020]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:14.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:14 smithi196 ceph-mon[67089]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:16.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:16 smithi144 ceph-mon[66781]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:16.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:16 smithi132 ceph-mon[32020]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:16.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:16 smithi196 ceph-mon[67089]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:16.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.955+0000 7fb3b4829700 1 -- 172.21.15.144:0/2764341154 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb3b00fe8f0 con 0x7fb39c0a2800 2024-03-18T23:02:16.957 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-03-18T23:02:16.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 msgr2=0x7fb39c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:16.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 0x7fb39c0a4cc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0005950 tx=0x7fb3a00058e0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 msgr2=0x7fb3b019fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:16.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006070 tx=0x7fb3a8007dc0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 shutdown_connections 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb39c0a2800 0x7fb39c0a4cc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3b0102de0 0x7fb3b019fb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3b0101240 0x7fb3b0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 --2- 172.21.15.144:0/2764341154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3b0102440 0x7fb3b0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 >> 172.21.15.144:0/2764341154 conn(0x7fb3b00fc800 msgr2=0x7fb3b00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 shutdown_connections 2024-03-18T23:02:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:02:16.958+0000 7fb3b582b700 1 -- 172.21.15.144:0/2764341154 wait complete. 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:17.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:17.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:17.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:17.565 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.1.service 2024-03-18T23:02:17.568 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-03-18T23:02:17.568 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:17.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:17.801 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:02:18.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:18 smithi196 ceph-mon[67089]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:18.218 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:18 smithi196 ceph-mon[67089]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:18.219 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:18 smithi196 ceph-mon[67089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:18 smithi144 ceph-mon[66781]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:18 smithi144 ceph-mon[66781]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:18 smithi144 ceph-mon[66781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:18 smithi132 ceph-mon[32020]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:18 smithi132 ceph-mon[32020]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:18 smithi132 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T23:02:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:02:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:02:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:02:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:19 smithi144 ceph-mon[66781]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:02:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:19 smithi196 ceph-mon[67089]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: from='osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:02:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:19 smithi132 ceph-mon[32020]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112016 s, 936 MB/s 2024-03-18T23:02:19.629 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-18T23:02:20.172 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:20 smithi196 ceph-mon[67089]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:20.172 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.172 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.172 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:20 smithi196 ceph-mon[67089]: from='osd.1 ' entity='osd.1' 2024-03-18T23:02:20.172 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-03-18T23:02:20.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-mon[66781]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:20.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.263 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-mon[66781]: from='osd.1 ' entity='osd.1' 2024-03-18T23:02:20.264 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.264 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1[71357]: 2024-03-18T23:02:20.061+0000 7f0b0e1ec700 -1 osd.1 0 waiting for initial osdmap 2024-03-18T23:02:20.264 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1[71357]: 2024-03-18T23:02:20.156+0000 7f0b05fd7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:02:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:20 smithi132 ceph-mon[32020]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:20 smithi132 ceph-mon[32020]: from='osd.1 ' entity='osd.1' 2024-03-18T23:02:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:20.409 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:02:21.392 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi144 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:21.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:21.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:21.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:02:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:02:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi144 2024-03-18T23:02:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi144 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T23:02:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:21.842 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 -- 172.21.15.196:0/932648312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 msgr2=0x7fd68c10a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 --2- 172.21.15.196:0/932648312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c10a7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009990 tx=0x7fd67c009ca0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 -- 172.21.15.196:0/932648312 shutdown_connections 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 --2- 172.21.15.196:0/932648312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c102360 0x7fd68c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 --2- 172.21.15.196:0/932648312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c10a7b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 --2- 172.21.15.196:0/932648312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c103560 0x7fd68c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.839+0000 7fd693d81700 1 -- 172.21.15.196:0/932648312 >> 172.21.15.196:0/932648312 conn(0x7fd68c0fd920 msgr2=0x7fd68c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- 172.21.15.196:0/932648312 shutdown_connections 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- 172.21.15.196:0/932648312 wait complete. 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 Processor -- start 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- start start 2024-03-18T23:02:21.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 0x7fd68c0760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103560 0x7fd68c074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c10df30 con 0x7fd68c102360 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c10ddb0 con 0x7fd68c103560 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.840+0000 7fd693d81700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c10e0b0 con 0x7fd68c103f00 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:33054/0 (socket says 172.21.15.196:33054) 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 -- 172.21.15.196:0/1513845757 learned_addr learned my addr 172.21.15.196:0/1513845757 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69131c700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103560 0x7fd68c074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd691b1d700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 0x7fd68c0760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 -- 172.21.15.196:0/1513845757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103560 msgr2=0x7fd68c074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103560 0x7fd68c074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.844 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 -- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 msgr2=0x7fd68c0760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 0x7fd68c0760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 -- 172.21.15.196:0/1513845757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c0754f0 con 0x7fd68c103f00 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd691b1d700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 0x7fd68c0760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd69231e700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0039c0 tx=0x7fd67c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c003680 con 0x7fd68c103f00 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c004150 con 0x7fd68c103f00 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1a25e0 con 0x7fd68c103f00 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0054a0 con 0x7fd68c103f00 2024-03-18T23:02:21.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.841+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a29d0 con 0x7fd68c103f00 2024-03-18T23:02:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.842+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c068fd0 con 0x7fd68c103f00 2024-03-18T23:02:21.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.843+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fd67c029060 con 0x7fd68c103f00 2024-03-18T23:02:21.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.844+0000 7fd68affd700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 0x7fd6740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:21.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.844+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fd67c0cdff0 con 0x7fd68c103f00 2024-03-18T23:02:21.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.844+0000 7fd691b1d700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 0x7fd6740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:21.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.845+0000 7fd691b1d700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 0x7fd6740a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1033c0 tx=0x7fd684009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:21.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:21.848+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd67c05d740 con 0x7fd68c103f00 2024-03-18T23:02:22.053 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:22.053+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd68c062750 con 0x7fd6740a2800 2024-03-18T23:02:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] boot 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:22.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] boot 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: osd.1 [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] boot 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:22.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:23 smithi196 ceph-mon[67089]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:02:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:23 smithi196 ceph-mon[67089]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:23 smithi144 ceph-mon[66781]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:02:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:23 smithi144 ceph-mon[66781]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:23 smithi132 ceph-mon[32020]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T23:02:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:23 smithi132 ceph-mon[32020]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:24 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/1439068372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:24 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:24.947 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:24 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]': finished 2024-03-18T23:02:24.948 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:24 smithi196 ceph-mon[67089]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:02:24.948 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:24 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:24 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/1439068372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:24 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:24 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]': finished 2024-03-18T23:02:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:24 smithi144 ceph-mon[66781]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:02:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:24 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:25.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:24 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/1439068372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:25.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:24 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]: dispatch 2024-03-18T23:02:25.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:24 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2adeb4b3-32ca-446e-9801-d535dd74c2d9"}]': finished 2024-03-18T23:02:25.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:24 smithi132 ceph-mon[32020]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T23:02:25.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:24 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:25.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:25 smithi196 ceph-mon[67089]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:25.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:25 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/3221755031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:25 smithi144 ceph-mon[66781]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:25 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/3221755031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:25 smithi132 ceph-mon[32020]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:25 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/3221755031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:28 smithi144 ceph-mon[66781]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:28 smithi132 ceph-mon[32020]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:28.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:28 smithi196 ceph-mon[67089]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:30.155 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:30 smithi196 ceph-mon[67089]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:30 smithi144 ceph-mon[66781]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:30.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:30 smithi132 ceph-mon[32020]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:32.298 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:32 smithi196 ceph-mon[67089]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:32.299 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:02:32.299 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:32 smithi144 ceph-mon[66781]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:02:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:32 smithi132 ceph-mon[32020]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T23:02:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:33.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:33 smithi196 ceph-mon[67089]: Deploying daemon osd.2 on smithi196 2024-03-18T23:02:33.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:33 smithi144 ceph-mon[66781]: Deploying daemon osd.2 on smithi196 2024-03-18T23:02:33.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:33 smithi132 ceph-mon[32020]: Deploying daemon osd.2 on smithi196 2024-03-18T23:02:34.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:34 smithi196 ceph-mon[67089]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:34 smithi144 ceph-mon[66781]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:34.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:34 smithi132 ceph-mon[32020]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:36.166 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:36 smithi196 ceph-mon[67089]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:36 smithi144 ceph-mon[66781]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:36 smithi132 ceph-mon[32020]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:37 smithi196 ceph-mon[67089]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:37 smithi144 ceph-mon[66781]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:37 smithi132 ceph-mon[32020]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:40.018 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-03-18T23:02:40.018 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.016+0000 7fd68affd700 1 -- 172.21.15.196:0/1513845757 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd68c062750 con 0x7fd6740a2800 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 msgr2=0x7fd6740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 0x7fd6740a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd68c1033c0 tx=0x7fd684009040 comp rx=0 tx=0).stop 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 msgr2=0x7fd68c074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0039c0 tx=0x7fd67c009580 comp rx=0 tx=0).stop 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 shutdown_connections 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fd6740a2800 0x7fd6740a4cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.020 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd68c103f00 0x7fd68c074c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd68c102360 0x7fd68c0760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 --2- 172.21.15.196:0/1513845757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd68c103560 0x7fd68c074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:40.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 >> 172.21.15.196:0/1513845757 conn(0x7fd68c0fd920 msgr2=0x7fd68c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:40.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 shutdown_connections 2024-03-18T23:02:40.021 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:02:40.019+0000 7fd693d81700 1 -- 172.21.15.196:0/1513845757 wait complete. 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:40.304 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.581 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.2.service 2024-03-18T23:02:40.584 INFO:tasks.cephadm:Deploying osd.3 on smithi132 with /dev/vg_nvme/lv_3... 2024-03-18T23:02:40.584 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:02:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:02:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:40.839 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:02:40.657+0000 7f36f97827c0 -1 osd.2 0 log_to_monitors true 2024-03-18T23:02:40.847 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:02:41.171 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:41 smithi132 ceph-mon[32020]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:41.172 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:41 smithi132 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:41.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:41 smithi196 ceph-mon[67089]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:41.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:41 smithi196 ceph-mon[67089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:41 smithi144 ceph-mon[66781]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:41 smithi144 ceph-mon[66781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T23:02:42.453 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:42.453 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:02:42.453 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.453 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:02:42.453 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:42.454 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:42 smithi132 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.474 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:02:42.153+0000 7f36eede7700 -1 osd.2 0 waiting for initial osdmap 2024-03-18T23:02:42.474 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:02:42.157+0000 7f36eabda700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:42 smithi196 ceph-mon[67089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: from='osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:42 smithi144 ceph-mon[66781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150968 s, 695 MB/s 2024-03-18T23:02:42.819 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:43.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi196 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 17080M 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.433 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.458 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi196 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 17080M 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:43.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T23:02:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi196 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 17080M 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:43.734 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:02:44.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:44 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:02:44.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:44 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:02:44.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:44 smithi132 ceph-mon[32020]: osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] boot 2024-03-18T23:02:44.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:44 smithi132 ceph-mon[32020]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:02:44.388 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:44 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:02:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:44 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:02:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:44 smithi196 ceph-mon[67089]: osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] boot 2024-03-18T23:02:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:44 smithi196 ceph-mon[67089]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:02:44.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:44 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:44 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:02:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:44 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:02:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:44 smithi144 ceph-mon[66781]: osd.2 [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] boot 2024-03-18T23:02:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:44 smithi144 ceph-mon[66781]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T23:02:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:44 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:02:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 -- 172.21.15.132:0/122325804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac102440 msgr2=0x7fe8ac1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/122325804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac102440 0x7fe8ac1028a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098f0 tx=0x7fe89c009c00 comp rx=0 tx=0).stop 2024-03-18T23:02:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 -- 172.21.15.132:0/122325804 shutdown_connections 2024-03-18T23:02:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/122325804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac101240 0x7fe8ac101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.187 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/122325804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/122325804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac102440 0x7fe8ac1028a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 -- 172.21.15.132:0/122325804 >> 172.21.15.132:0/122325804 conn(0x7fe8ac0fc7e0 msgr2=0x7fe8ac0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:02:45.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 -- 172.21.15.132:0/122325804 shutdown_connections 2024-03-18T23:02:45.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.186+0000 7fe8b0a49700 1 -- 172.21.15.132:0/122325804 wait complete. 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 Processor -- start 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 -- start start 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac102440 0x7fe8ac103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10ce30 con 0x7fe8ac102de0 2024-03-18T23:02:45.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10ccb0 con 0x7fe8ac102440 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8b0a49700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac10cfb0 con 0x7fe8ac101240 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:48970/0 (socket says 172.21.15.132:48970) 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac103d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58154/0 (socket says 172.21.15.132:58154) 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8aa59c700 1 -- 172.21.15.132:0/4289526209 learned_addr learned my addr 172.21.15.132:0/4289526209 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.187+0000 7fe8a9d9b700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac102440 0x7fe8ac103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 -- 172.21.15.132:0/4289526209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac102440 msgr2=0x7fe8ac103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac102440 0x7fe8ac103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 -- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 msgr2=0x7fe8ac103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 -- 172.21.15.132:0/4289526209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac104610 con 0x7fe8ac101240 2024-03-18T23:02:45.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8aa59c700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e950 tx=0x7fe8a000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0004230 con 0x7fe8ac101240 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a00043d0 con 0x7fe8ac101240 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac1a27d0 con 0x7fe8ac101240 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0003dd0 con 0x7fe8ac101240 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.188+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a2c90 con 0x7fe8ac101240 2024-03-18T23:02:45.192 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.189+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac068fd0 con 0x7fe8ac101240 2024-03-18T23:02:45.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.190+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fe8a000c7e0 con 0x7fe8ac101240 2024-03-18T23:02:45.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.191+0000 7fe89b7fe700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 0x7fe8940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:02:45.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.191+0000 7fe8a9d9b700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 0x7fe8940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:02:45.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.191+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fe8a00cd810 con 0x7fe8ac101240 2024-03-18T23:02:45.193 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.191+0000 7fe8a9d9b700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 0x7fe8940a4cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098c0 tx=0x7fe89c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:02:45.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.195+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe8a009be40 con 0x7fe8ac101240 2024-03-18T23:02:45.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:45.404+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe8ac06a700 con 0x7fe8940a2800 2024-03-18T23:02:45.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:45 smithi144 ceph-mon[66781]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:02:45.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:45 smithi132 ceph-mon[32020]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:02:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:45 smithi196 ceph-mon[67089]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:46.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:46.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:46.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:46 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:46.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:46.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:46 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:02:46.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:02:46.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:46 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:46.942 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:46.942 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:46.942 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:46.942 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51187]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:47.255 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:47.256 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:47.256 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51190]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:47.256 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:02:46 smithi132 sudo[51190]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:47.463 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:47.463 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:47 smithi144 ceph-mon[66781]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:02:47.463 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:47.463 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:47 smithi144 ceph-mon[66781]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:47.464 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:47.464 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:47.464 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75517]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:47.464 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75517]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:47.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:47.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:47 smithi132 ceph-mon[32020]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:02:47.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:47.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:47 smithi132 ceph-mon[32020]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:47.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:02:47.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:47 smithi196 ceph-mon[67089]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T23:02:47.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:47.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:47 smithi196 ceph-mon[67089]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:47.859 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75520]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:47.859 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:47.859 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75520]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:47.859 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:02:47 smithi144 sudo[75520]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:47.994 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76044]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:47.994 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76044]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:47.994 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76044]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:47.994 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76044]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:48.473 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76048]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:48.474 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76048]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:48.474 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76048]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:48.474 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:02:47 smithi196 sudo[76048]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:48.517 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51265]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:48.517 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51265]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:48.517 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51265]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:48.517 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51265]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:48.778 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 ceph-mon[67089]: osdmap e22: 3 total, 3 up, 3 in 2024-03-18T23:02:48.778 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:48.778 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:48.778 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 sudo[76051]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:48.779 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 sudo[76051]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:48.779 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 sudo[76051]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:48.779 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:48 smithi196 sudo[76051]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:48.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:48 smithi144 ceph-mon[66781]: osdmap e22: 3 total, 3 up, 3 in 2024-03-18T23:02:48.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:48.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:48.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:48.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:48.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:48.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 sudo[51268]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:48.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 ceph-mon[32020]: osdmap e22: 3 total, 3 up, 3 in 2024-03-18T23:02:48.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:48.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:02:49.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:02:49.028+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fe8a0097db0 con 0x7fe8ac101240 2024-03-18T23:02:49.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 sudo[76054]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:49.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 sudo[76054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:49.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 sudo[76054]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:49.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 sudo[76054]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:49.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75523]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T23:02:49.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75523]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:49.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75523]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:49.509 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75523]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75526]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T23:02:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T23:02:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75526]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T23:02:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 sudo[75526]: pam_unix(sudo:session): session closed for user root 2024-03-18T23:02:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]': finished 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:02:49.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2060452628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:49.861 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:49 smithi144 ceph-mon[66781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]': finished 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:02:49.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2060452628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:49.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:49 smithi132 ceph-mon[32020]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/592495469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "253c2374-3879-4485-8ecb-a893a210b624"}]': finished 2024-03-18T23:02:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2060452628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:02:49.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:49 smithi196 ceph-mon[67089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T23:02:52.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:52 smithi144 ceph-mon[66781]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:52.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:52 smithi132 ceph-mon[32020]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:52.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:52 smithi196 ceph-mon[67089]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:02:54.355 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:54 smithi132 ceph-mon[32020]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 232 KiB/s wr, 16 op/s 2024-03-18T23:02:54.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:54 smithi144 ceph-mon[66781]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 232 KiB/s wr, 16 op/s 2024-03-18T23:02:54.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:54 smithi196 ceph-mon[67089]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 232 KiB/s wr, 16 op/s 2024-03-18T23:02:56.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:56 smithi144 ceph-mon[66781]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-18T23:02:56.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:02:56.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:56 smithi132 ceph-mon[32020]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-18T23:02:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:02:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:56.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:56 smithi196 ceph-mon[67089]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-03-18T23:02:56.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T23:02:56.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:02:57.332 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:57 smithi196 ceph-mon[67089]: Deploying daemon osd.3 on smithi132 2024-03-18T23:02:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:57 smithi144 ceph-mon[66781]: Deploying daemon osd.3 on smithi132 2024-03-18T23:02:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:57 smithi132 ceph-mon[32020]: Deploying daemon osd.3 on smithi132 2024-03-18T23:02:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:02:58 smithi196 ceph-mon[67089]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 135 KiB/s wr, 9 op/s 2024-03-18T23:02:58.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:02:58 smithi144 ceph-mon[66781]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 135 KiB/s wr, 9 op/s 2024-03-18T23:02:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:02:58 smithi132 ceph-mon[32020]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 135 KiB/s wr, 9 op/s 2024-03-18T23:03:00.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:00 smithi144 ceph-mon[66781]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-03-18T23:03:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:00 smithi132 ceph-mon[32020]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-03-18T23:03:00.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:00 smithi196 ceph-mon[67089]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-03-18T23:03:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:02 smithi144 ceph-mon[66781]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:02 smithi132 ceph-mon[32020]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:02.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:02 smithi196 ceph-mon[67089]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:02.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:02.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:04.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:04.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.779+0000 7fe89b7fe700 1 -- 172.21.15.132:0/4289526209 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8ac06a700 con 0x7fe8940a2800 2024-03-18T23:03:04.780 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 3 on host 'smithi132' 2024-03-18T23:03:04.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 msgr2=0x7fe8940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:04.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 0x7fe8940a4cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0098c0 tx=0x7fe89c01a040 comp rx=0 tx=0).stop 2024-03-18T23:03:04.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 msgr2=0x7fe8ac1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e950 tx=0x7fe8a000ec60 comp rx=0 tx=0).stop 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 shutdown_connections 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe8940a2800 0x7fe8940a4cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8ac102de0 0x7fe8ac103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac101240 0x7fe8ac1051d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 --2- 172.21.15.132:0/4289526209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ac102440 0x7fe8ac103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.781+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 >> 172.21.15.132:0/4289526209 conn(0x7fe8ac0fc7e0 msgr2=0x7fe8ac069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.782+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 shutdown_connections 2024-03-18T23:03:04.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:04.782+0000 7fe8b0a49700 1 -- 172.21.15.132:0/4289526209 wait complete. 2024-03-18T23:03:05.425 DEBUG:teuthology.orchestra.run.smithi132:osd.3> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.3.service 2024-03-18T23:03:05.427 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_3... 2024-03-18T23:03:05.427 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:03:05.674 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:03:05.953 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:05 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:05.953 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:05 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:05.953 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:05 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:05.954 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:05 smithi132 ceph-mon[32020]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:05.954 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:05 smithi132 ceph-mon[32020]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:03:06.051 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:05 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:06.051 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:05 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:06.051 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:05 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:06.052 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:05 smithi144 ceph-mon[66781]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:06.052 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:05 smithi144 ceph-mon[66781]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:03:06.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:05 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:06.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:05 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:06.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:05 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:06.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:05 smithi196 ceph-mon[67089]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:06.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:05 smithi196 ceph-mon[67089]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T23:03:06.951 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-mon[32020]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:03:06.951 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-mon[32020]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:03:06.951 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-mon[32020]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:03:06.951 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:06.952 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:03:06.768+0000 7fc6179d8700 -1 osd.3 0 waiting for initial osdmap 2024-03-18T23:03:06.952 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:03:06 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:03:06.774+0000 7fc6137cb700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:06.952 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:06 smithi144 ceph-mon[66781]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:03:06.952 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:06 smithi144 ceph-mon[66781]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:03:06.952 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:06 smithi144 ceph-mon[66781]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:03:06.952 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:06 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:06 smithi196 ceph-mon[67089]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T23:03:07.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:06 smithi196 ceph-mon[67089]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T23:03:07.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:06 smithi196 ceph-mon[67089]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:03:07.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:06 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.559 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:03:07.559 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:03:07.559 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-18T23:03:07.560 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-18T23:03:07.560 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011047 s, 949 MB/s 2024-03-18T23:03:07.560 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-18T23:03:07.854 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:07 smithi132 ceph-mon[32020]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:03:07.855 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:07 smithi132 ceph-mon[32020]: osdmap e25: 4 total, 3 up, 4 in 2024-03-18T23:03:07.855 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.855 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.855 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:07 smithi132 ceph-mon[32020]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:07.919 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:07 smithi144 ceph-mon[66781]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:03:07.919 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:07 smithi144 ceph-mon[66781]: osdmap e25: 4 total, 3 up, 4 in 2024-03-18T23:03:07.919 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.919 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:07.920 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:07 smithi144 ceph-mon[66781]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:08.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-03-18T23:03:08.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:07 smithi196 ceph-mon[67089]: from='osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:03:08.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:07 smithi196 ceph-mon[67089]: osdmap e25: 4 total, 3 up, 4 in 2024-03-18T23:03:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:07 smithi196 ceph-mon[67089]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-18T23:03:08.438 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:03:08.839 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] boot 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:08.840 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:08.841 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:08.841 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:08.841 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:08 smithi144 ceph-mon[66781]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] boot 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:09.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:09.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:09.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.117 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:08 smithi132 ceph-mon[32020]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: osd.3 [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] boot 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:09.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:09.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:09.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:09.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:09.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:08 smithi196 ceph-mon[67089]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T23:03:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 -- 172.21.15.144:0/1859513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 msgr2=0x7fdc0c102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 --2- 172.21.15.144:0/1859513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c102780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc080099c0 tx=0x7fdc08009cd0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 -- 172.21.15.144:0/1859513006 shutdown_connections 2024-03-18T23:03:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 --2- 172.21.15.144:0/1859513006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c102780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 --2- 172.21.15.144:0/1859513006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc0c103f00 0x7fdc0c10a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 --2- 172.21.15.144:0/1859513006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 -- 172.21.15.144:0/1859513006 >> 172.21.15.144:0/1859513006 conn(0x7fdc0c0fd920 msgr2=0x7fdc0c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 -- 172.21.15.144:0/1859513006 shutdown_connections 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.973+0000 7fdc13d02700 1 -- 172.21.15.144:0/1859513006 wait complete. 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 Processor -- start 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 -- start start 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc0c103f00 0x7fdc0c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10e020 con 0x7fdc0c103560 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10dea0 con 0x7fdc0c102360 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc13d02700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c10e1a0 con 0x7fdc0c103f00 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:09.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39932/0 (socket says 172.21.15.144:39932) 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 -- 172.21.15.144:0/3026336587 learned_addr learned my addr 172.21.15.144:0/3026336587 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc1129d700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 -- 172.21.15.144:0/3026336587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc0c103f00 msgr2=0x7fdc0c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc0c103f00 0x7fdc0c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 -- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 msgr2=0x7fdc0c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 -- 172.21.15.144:0/3026336587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c1a0320 con 0x7fdc0c102360 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc1129d700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:03:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.974+0000 7fdc11a9e700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c010 tx=0x7fdc08003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.975+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08027070 con 0x7fdc0c102360 2024-03-18T23:03:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.975+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1a05b0 con 0x7fdc0c102360 2024-03-18T23:03:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.975+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc08022470 con 0x7fdc0c102360 2024-03-18T23:03:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.975+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c1a0b60 con 0x7fdc0c102360 2024-03-18T23:03:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.975+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800f6a0 con 0x7fdc0c102360 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.976+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c068fd0 con 0x7fdc0c102360 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.977+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fdc0802a070 con 0x7fdc0c102360 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.977+0000 7fdc02ffd700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 0x7fdbf80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.978+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fdc080cecb0 con 0x7fdc0c102360 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.978+0000 7fdc1129d700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 0x7fdbf80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.978+0000 7fdc1129d700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 0x7fdbf80a4d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc007900 tx=0x7fdbfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:09.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:09.981+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc08013070 con 0x7fdc0c102360 2024-03-18T23:03:10.035 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:09 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi132 2024-03-18T23:03:10.035 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:09 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 8544M 2024-03-18T23:03:10.035 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:09 smithi132 ceph-mon[32020]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:10.035 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:09 smithi132 ceph-mon[32020]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:03:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:09 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi132 2024-03-18T23:03:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:09 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 8544M 2024-03-18T23:03:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:09 smithi144 ceph-mon[66781]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:10.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:09 smithi144 ceph-mon[66781]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:03:10.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:10.186+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc0c062750 con 0x7fdbf80a2890 2024-03-18T23:03:10.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:09 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi132 2024-03-18T23:03:10.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:09 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 8544M 2024-03-18T23:03:10.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:09 smithi196 ceph-mon[67089]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:10.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:09 smithi196 ceph-mon[67089]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T23:03:11.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:11.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:10 smithi144 ceph-mon[66781]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:03:11.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:11.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:11.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:11.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:10 smithi132 ceph-mon[32020]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:03:11.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:11.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:11.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:11.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:10 smithi196 ceph-mon[67089]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T23:03:12.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:11 smithi144 ceph-mon[66781]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:12.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:11 smithi144 ceph-mon[66781]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:12.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:11 smithi144 ceph-mon[66781]: osdmap e30: 4 total, 4 up, 4 in 2024-03-18T23:03:12.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:11 smithi132 ceph-mon[32020]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:12.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:11 smithi132 ceph-mon[32020]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:12.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:11 smithi132 ceph-mon[32020]: osdmap e30: 4 total, 4 up, 4 in 2024-03-18T23:03:12.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:11 smithi196 ceph-mon[67089]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:12.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:11 smithi196 ceph-mon[67089]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:12.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:11 smithi196 ceph-mon[67089]: osdmap e30: 4 total, 4 up, 4 in 2024-03-18T23:03:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:12 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/3100754767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:12 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:12 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]': finished 2024-03-18T23:03:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:12 smithi144 ceph-mon[66781]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:03:13.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:13.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:12 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/3100754767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:12 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:12 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]': finished 2024-03-18T23:03:13.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:12 smithi132 ceph-mon[32020]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:03:13.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:13.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:12 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/3100754767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:12 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]: dispatch 2024-03-18T23:03:13.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:12 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e8564c1-d17a-453d-a5e2-af4740fb0d93"}]': finished 2024-03-18T23:03:13.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:12 smithi196 ceph-mon[67089]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T23:03:13.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:13 smithi144 ceph-mon[66781]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:03:14.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/2694552351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:14.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:13 smithi132 ceph-mon[32020]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:03:14.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:13 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/2694552351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:14.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:13 smithi196 ceph-mon[67089]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T23:03:14.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:13 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/2694552351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:16.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:16 smithi144 ceph-mon[66781]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-18T23:03:16.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:16 smithi132 ceph-mon[32020]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-18T23:03:16.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:16 smithi196 ceph-mon[67089]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-18T23:03:18.232 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:18 smithi144 ceph-mon[66781]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:03:18.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:18 smithi132 ceph-mon[32020]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:03:18.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:18 smithi196 ceph-mon[67089]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:03:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:20 smithi144 ceph-mon[66781]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:03:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:03:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:20 smithi132 ceph-mon[32020]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:03:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:03:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:20 smithi196 ceph-mon[67089]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:03:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T23:03:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:21.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:21 smithi144 ceph-mon[66781]: Deploying daemon osd.4 on smithi144 2024-03-18T23:03:21.327 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:21 smithi132 ceph-mon[32020]: Deploying daemon osd.4 on smithi144 2024-03-18T23:03:21.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:21 smithi196 ceph-mon[67089]: Deploying daemon osd.4 on smithi144 2024-03-18T23:03:22.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:22 smithi144 ceph-mon[66781]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:03:22.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:22 smithi132 ceph-mon[32020]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:03:22.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:22 smithi196 ceph-mon[67089]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:03:24.282 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:24 smithi144 ceph-mon[66781]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:24.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:24 smithi132 ceph-mon[32020]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:24 smithi196 ceph-mon[67089]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:26.135 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:26 smithi144 ceph-mon[66781]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:26.135 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.135 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.135 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:26.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:26 smithi132 ceph-mon[32020]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:26.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:26 smithi196 ceph-mon[67089]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:28.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:28.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:28.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:28.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.699+0000 7fdc02ffd700 1 -- 172.21.15.144:0/3026336587 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc0c062750 con 0x7fdbf80a2890 2024-03-18T23:03:28.700 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-03-18T23:03:28.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 msgr2=0x7fdbf80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:28.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 0x7fdbf80a4d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc007900 tx=0x7fdbfc008040 comp rx=0 tx=0).stop 2024-03-18T23:03:28.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 msgr2=0x7fdc0c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:28.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c010 tx=0x7fdc08003680 comp rx=0 tx=0).stop 2024-03-18T23:03:28.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 shutdown_connections 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdbf80a2890 0x7fdbf80a4d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc0c102360 0x7fdc0c1986c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc0c103f00 0x7fdc0c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 --2- 172.21.15.144:0/3026336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc0c103560 0x7fdc0c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 >> 172.21.15.144:0/3026336587 conn(0x7fdc0c0fd920 msgr2=0x7fdc0c106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 shutdown_connections 2024-03-18T23:03:28.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:03:28.701+0000 7fdc13d02700 1 -- 172.21.15.144:0/3026336587 wait complete. 2024-03-18T23:03:29.335 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.4.service 2024-03-18T23:03:29.338 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-03-18T23:03:29.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T23:03:29.583 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:29.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:29 smithi196 ceph-mon[67089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:30.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:29 smithi144 ceph-mon[66781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:30.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:30.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:30.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:30.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:30.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:30.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:29 smithi132 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T23:03:30.777 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-mon[66781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:03:30.777 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-mon[66781]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:30.778 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-mon[66781]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:03:30.778 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:30.778 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-mon[66781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:30.778 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:03:30.695+0000 7f8e28d06700 -1 osd.4 0 waiting for initial osdmap 2024-03-18T23:03:30.778 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:03:30 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:03:30.703+0000 7f8e24af9700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:30.903 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:30 smithi196 ceph-mon[67089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:03:30.904 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:30 smithi196 ceph-mon[67089]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:30.904 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:30 smithi196 ceph-mon[67089]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:03:30.904 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:30 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:30.904 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:30 smithi196 ceph-mon[67089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:31.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:30 smithi132 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T23:03:31.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:30 smithi132 ceph-mon[32020]: from='osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:31.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:30 smithi132 ceph-mon[32020]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T23:03:31.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:30 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:31.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:30 smithi132 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112646 s, 931 MB/s 2024-03-18T23:03:31.273 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-18T23:03:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:31 smithi196 ceph-mon[67089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:03:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:31 smithi196 ceph-mon[67089]: osdmap e33: 5 total, 4 up, 5 in 2024-03-18T23:03:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:31 smithi196 ceph-mon[67089]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:31 smithi144 ceph-mon[66781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:03:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:31 smithi144 ceph-mon[66781]: osdmap e33: 5 total, 4 up, 5 in 2024-03-18T23:03:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:31 smithi144 ceph-mon[66781]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:31.870 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-03-18T23:03:32.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:31 smithi132 ceph-mon[32020]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:03:32.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:31 smithi132 ceph-mon[32020]: osdmap e33: 5 total, 4 up, 5 in 2024-03-18T23:03:32.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:32.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:32.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:31 smithi132 ceph-mon[32020]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-18T23:03:32.117 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] boot 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:32.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:32.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:32 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:03:33.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:03:33.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] boot 2024-03-18T23:03:33.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:33.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:32 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:03:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:03:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: osd.4 [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] boot 2024-03-18T23:03:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T23:03:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:33.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:32 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:33.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/2304566561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54102de0 msgr2=0x7f8e54109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/2304566561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54102de0 0x7f8e54109690 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00b0a0 tx=0x7f8e4c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/2304566561 shutdown_connections 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/2304566561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e54101240 0x7f8e54101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/2304566561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54102de0 0x7f8e54109690 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/2304566561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/2304566561 >> 172.21.15.196:0/2304566561 conn(0x7f8e540fc800 msgr2=0x7f8e540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/2304566561 shutdown_connections 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.617+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/2304566561 wait complete. 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 Processor -- start 2024-03-18T23:03:33.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 -- start start 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e5419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e54102de0 0x7f8e5419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410ce20 con 0x7f8e54102440 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410cca0 con 0x7f8e54102de0 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5aaf3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410cfa0 con 0x7f8e54101240 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5888f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5888f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:35396/0 (socket says 172.21.15.196:35396) 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.618+0000 7f8e5888f700 1 -- 172.21.15.196:0/667164686 learned_addr learned my addr 172.21.15.196:0/667164686 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e53fff700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e5419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 -- 172.21.15.196:0/667164686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e54102de0 msgr2=0x7f8e5419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e54102de0 0x7f8e5419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 -- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 msgr2=0x7f8e5419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e5419d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 -- 172.21.15.196:0/667164686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e541a0110 con 0x7f8e54101240 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e53fff700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e5419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5888f700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800c8f0 tx=0x7f8e4800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48004220 con 0x7f8e54101240 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48007de0 con 0x7f8e54101240 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e541a0400 con 0x7f8e54101240 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48003de0 con 0x7f8e54101240 2024-03-18T23:03:33.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.619+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e541a0940 con 0x7f8e54101240 2024-03-18T23:03:33.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.620+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e54068fd0 con 0x7f8e54101240 2024-03-18T23:03:33.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.621+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f8e48007900 con 0x7f8e54101240 2024-03-18T23:03:33.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.622+0000 7f8e51ffb700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 0x7f8e440a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:33.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.622+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f8e480cd630 con 0x7f8e54101240 2024-03-18T23:03:33.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.622+0000 7f8e53fff700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 0x7f8e440a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:33.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.622+0000 7f8e53fff700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 0x7f8e440a4dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:33.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.626+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e48014030 con 0x7f8e54101240 2024-03-18T23:03:33.833 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:33.833+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8e54062750 con 0x7f8e440a2910 2024-03-18T23:03:33.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:33 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi144 2024-03-18T23:03:33.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:33 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-18T23:03:33.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:33 smithi196 ceph-mon[67089]: osdmap e35: 5 total, 5 up, 5 in 2024-03-18T23:03:33.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:33 smithi196 ceph-mon[67089]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:34.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:33 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi144 2024-03-18T23:03:34.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:33 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-18T23:03:34.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:33 smithi144 ceph-mon[66781]: osdmap e35: 5 total, 5 up, 5 in 2024-03-18T23:03:34.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:33 smithi144 ceph-mon[66781]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:34.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:33 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi144 2024-03-18T23:03:34.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:33 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-18T23:03:34.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:33 smithi132 ceph-mon[32020]: osdmap e35: 5 total, 5 up, 5 in 2024-03-18T23:03:34.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:33 smithi132 ceph-mon[32020]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:34.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:34 smithi196 ceph-mon[67089]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:34.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:34.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:34.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:35.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:34 smithi144 ceph-mon[66781]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:35.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:35.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:35.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:34 smithi132 ceph-mon[32020]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:35.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:35.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:35 smithi196 ceph-mon[67089]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:36.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:35 smithi144 ceph-mon[66781]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:36.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:35 smithi132 ceph-mon[32020]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:36.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:36 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/3726895950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:36.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:36 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:36.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:36 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]': finished 2024-03-18T23:03:36.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:36 smithi196 ceph-mon[67089]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:03:36.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:37.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:36 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/3726895950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:37.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:36 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:37.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:36 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]': finished 2024-03-18T23:03:37.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:36 smithi144 ceph-mon[66781]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:03:37.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:37.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:36 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/3726895950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:37.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:36 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]: dispatch 2024-03-18T23:03:37.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:36 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5eedd04e-128b-491d-82e6-1988d13b1145"}]': finished 2024-03-18T23:03:37.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:36 smithi132 ceph-mon[32020]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T23:03:37.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/3926613869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:38.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:37 smithi144 ceph-mon[66781]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/3926613869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:38.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:37 smithi132 ceph-mon[32020]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:38.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/3926613869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:03:38.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:37 smithi196 ceph-mon[67089]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:40 smithi144 ceph-mon[66781]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:40.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:40 smithi132 ceph-mon[32020]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:40 smithi196 ceph-mon[67089]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:42.326 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:42 smithi196 ceph-mon[67089]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:42.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:42 smithi144 ceph-mon[66781]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:42.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:42 smithi132 ceph-mon[32020]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:44 smithi144 ceph-mon[66781]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:44 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:03:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:44 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:44 smithi132 ceph-mon[32020]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:03:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:44.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:44 smithi196 ceph-mon[67089]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:44.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:44 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T23:03:44.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:44 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:45.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:45 smithi144 ceph-mon[66781]: Deploying daemon osd.5 on smithi196 2024-03-18T23:03:45.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:45 smithi132 ceph-mon[32020]: Deploying daemon osd.5 on smithi196 2024-03-18T23:03:45.429 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:45 smithi196 ceph-mon[67089]: Deploying daemon osd.5 on smithi196 2024-03-18T23:03:46.337 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:46 smithi196 ceph-mon[67089]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:46 smithi144 ceph-mon[66781]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:46.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:46 smithi132 ceph-mon[32020]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:48 smithi144 ceph-mon[66781]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:48 smithi132 ceph-mon[32020]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:48.415 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:48 smithi196 ceph-mon[67089]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:50.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:50.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:49 smithi196 ceph-mon[67089]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:49 smithi144 ceph-mon[66781]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:50.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:49 smithi132 ceph-mon[32020]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-03-18T23:03:52.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.147+0000 7f8e51ffb700 1 -- 172.21.15.196:0/667164686 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e54062750 con 0x7f8e440a2910 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.149+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 msgr2=0x7f8e440a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.149+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 0x7f8e440a4dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40005dc0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.149+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 msgr2=0x7f8e5406b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.149+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800c8f0 tx=0x7f8e4800ccb0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 shutdown_connections 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8e440a2910 0x7f8e440a4dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e54102de0 0x7f8e5419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54101240 0x7f8e5406b610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 --2- 172.21.15.196:0/667164686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e54102440 0x7f8e5419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:52.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 >> 172.21.15.196:0/667164686 conn(0x7f8e540fc800 msgr2=0x7f8e541056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:52.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 shutdown_connections 2024-03-18T23:03:52.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:03:52.150+0000 7f8e5aaf3700 1 -- 172.21.15.196:0/667164686 wait complete. 2024-03-18T23:03:52.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:52 smithi144 ceph-mon[66781]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:52.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:52 smithi132 ceph-mon[32020]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:52.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:52 smithi196 ceph-mon[67089]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:52.834 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.5.service 2024-03-18T23:03:52.836 INFO:tasks.cephadm:Deploying osd.6 on smithi132 with /dev/vg_nvme/lv_2... 2024-03-18T23:03:52.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:03:53.083 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:53.414 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:53 smithi132 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:53.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:53 smithi196 ceph-mon[67089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:53 smithi144 ceph-mon[66781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:54.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:54 smithi132 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:54.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:03:54.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:03:54.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:54.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:54 smithi196 ceph-mon[67089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: from='osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:54.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:54 smithi144 ceph-mon[66781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:03:54.996 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:03:54.996 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:03:54.996 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-18T23:03:54.997 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-18T23:03:54.997 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112958 s, 928 MB/s 2024-03-18T23:03:54.997 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 2024-03-18T23:03:54.997 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:55.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-mon[67089]: from='osd.5 ' entity='osd.5' 2024-03-18T23:03:55.474 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:03:55.167+0000 7fd740737700 -1 osd.5 0 waiting for initial osdmap 2024-03-18T23:03:55.475 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:03:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:03:55.301+0000 7fd738522700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:03:55.608 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:55 smithi132 ceph-mon[32020]: from='osd.5 ' entity='osd.5' 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:55.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:55.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:55 smithi144 ceph-mon[66781]: from='osd.5 ' entity='osd.5' 2024-03-18T23:03:55.636 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2024-03-18T23:03:55.909 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:03:56.312 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: osdmap e39: 6 total, 5 up, 6 in 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi196 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 8540M 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.313 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.314 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:03:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:03:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: osdmap e39: 6 total, 5 up, 6 in 2024-03-18T23:03:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi196 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 8540M 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: osdmap e39: 6 total, 5 up, 6 in 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi196 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 8540M 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:03:56.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- 172.21.15.132:0/1334161033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 msgr2=0x7f16e4101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- 172.21.15.132:0/1334161033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e4101670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099c0 tx=0x7f16e0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- 172.21.15.132:0/1334161033 shutdown_connections 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- 172.21.15.132:0/1334161033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102df0 0x7f16e41096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- 172.21.15.132:0/1334161033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e4101670 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- 172.21.15.132:0/1334161033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102450 0x7f16e41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- 172.21.15.132:0/1334161033 >> 172.21.15.132:0/1334161033 conn(0x7f16e40fc810 msgr2=0x7f16e40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- 172.21.15.132:0/1334161033 shutdown_connections 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- 172.21.15.132:0/1334161033 wait complete. 2024-03-18T23:03:57.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 Processor -- start 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- start start 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102450 0x7f16e419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102df0 0x7f16e419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.412+0000 7f16ec07f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410ce40 con 0x7f16e4101250 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16ec07f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410ccc0 con 0x7f16e4102df0 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16ec07f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410cfc0 con 0x7f16e4102450 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43538/0 (socket says 172.21.15.132:43538) 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 -- 172.21.15.132:0/858081764 learned_addr learned my addr 172.21.15.132:0/858081764 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e961a700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102450 0x7f16e419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16ea61c700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102df0 0x7f16e419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:57.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 -- 172.21.15.132:0/858081764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102450 msgr2=0x7f16e419d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102450 0x7f16e419d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 -- 172.21.15.132:0/858081764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102df0 msgr2=0x7f16e419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102df0 0x7f16e419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 -- 172.21.15.132:0/858081764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e41a01e0 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16e9e1b700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bff0 tx=0x7f16e0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0027070 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0004150 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0022c00 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e41a0470 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.413+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41a09a0 con 0x7f16e4101250 2024-03-18T23:03:57.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.414+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4068fd0 con 0x7f16e4101250 2024-03-18T23:03:57.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.415+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f16e002a070 con 0x7f16e4101250 2024-03-18T23:03:57.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.416+0000 7f16daffd700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 0x7f16d00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:03:57.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.416+0000 7f16e961a700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 0x7f16d00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:03:57.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.416+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f16e00cea10 con 0x7f16e4101250 2024-03-18T23:03:57.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.416+0000 7f16e961a700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 0x7f16d00a4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16e406bb50 tx=0x7f16d400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:03:57.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.421+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f16e005e740 con 0x7f16e4101250 2024-03-18T23:03:57.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:57 smithi144 ceph-mon[66781]: osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] boot 2024-03-18T23:03:57.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:57 smithi144 ceph-mon[66781]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:03:57.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:57.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:57 smithi144 ceph-mon[66781]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:57.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:57 smithi132 ceph-mon[32020]: osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] boot 2024-03-18T23:03:57.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:57 smithi132 ceph-mon[32020]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:03:57.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:57.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:57 smithi132 ceph-mon[32020]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:57.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:03:57.628+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f16e4062810 con 0x7f16d00a2910 2024-03-18T23:03:57.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:57 smithi196 ceph-mon[67089]: osd.5 [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] boot 2024-03-18T23:03:57.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:57 smithi196 ceph-mon[67089]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T23:03:57.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:03:57.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:57 smithi196 ceph-mon[67089]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-18T23:03:58.555 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:58 smithi132 ceph-mon[32020]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:03:58.555 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:58 smithi132 ceph-mon[32020]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:58.555 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:58.555 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:58.555 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:58 smithi144 ceph-mon[66781]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:03:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:58 smithi144 ceph-mon[66781]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:58.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:58 smithi196 ceph-mon[67089]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T23:03:58.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:58 smithi196 ceph-mon[67089]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:03:58.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:03:58.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:03:58.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:03:59.431 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:59 smithi132 ceph-mon[32020]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:03:59.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:59 smithi132 ceph-mon[32020]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T23:03:59.432 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:03:59 smithi132 ceph-mon[32020]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:59 smithi144 ceph-mon[66781]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:03:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:59 smithi144 ceph-mon[66781]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T23:03:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:03:59 smithi144 ceph-mon[66781]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:03:59.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:59 smithi196 ceph-mon[67089]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T23:03:59.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:59 smithi196 ceph-mon[67089]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T23:03:59.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:03:59 smithi196 ceph-mon[67089]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:00.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:00 smithi196 ceph-mon[67089]: osdmap e44: 6 total, 6 up, 6 in 2024-03-18T23:04:00.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]: dispatch 2024-03-18T23:04:00.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]': finished 2024-03-18T23:04:00.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:00 smithi196 ceph-mon[67089]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:04:00.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:00 smithi144 ceph-mon[66781]: osdmap e44: 6 total, 6 up, 6 in 2024-03-18T23:04:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]: dispatch 2024-03-18T23:04:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]': finished 2024-03-18T23:04:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:00 smithi144 ceph-mon[66781]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:04:01.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:01.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:00 smithi132 ceph-mon[32020]: osdmap e44: 6 total, 6 up, 6 in 2024-03-18T23:04:01.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]: dispatch 2024-03-18T23:04:01.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4062306971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2ea776-34b0-45ca-8bfa-f7e5428ce0c9"}]': finished 2024-03-18T23:04:01.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:00 smithi132 ceph-mon[32020]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T23:04:01.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:01.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/662460541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:01.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:01 smithi196 ceph-mon[67089]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:02.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/662460541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:02.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:01 smithi144 ceph-mon[66781]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:02.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/662460541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:02.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:01 smithi132 ceph-mon[32020]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:04 smithi144 ceph-mon[66781]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-18T23:04:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:04 smithi132 ceph-mon[32020]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-18T23:04:04.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:04 smithi196 ceph-mon[67089]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-18T23:04:06.332 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:06 smithi132 ceph-mon[32020]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:04:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:06 smithi144 ceph-mon[66781]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:04:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:06 smithi196 ceph-mon[67089]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.327 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:08 smithi132 ceph-mon[32020]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.327 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:04:08.327 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:08 smithi144 ceph-mon[66781]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:04:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:08 smithi196 ceph-mon[67089]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:04:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T23:04:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:09 smithi144 ceph-mon[66781]: Deploying daemon osd.6 on smithi132 2024-03-18T23:04:09.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:09 smithi132 ceph-mon[32020]: Deploying daemon osd.6 on smithi132 2024-03-18T23:04:09.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:09 smithi196 ceph-mon[67089]: Deploying daemon osd.6 on smithi132 2024-03-18T23:04:10.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:10 smithi144 ceph-mon[66781]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:04:10.361 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:10 smithi132 ceph-mon[32020]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:04:10.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:10 smithi196 ceph-mon[67089]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T23:04:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:12 smithi144 ceph-mon[66781]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:04:12.362 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:12 smithi132 ceph-mon[32020]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:04:12.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:12 smithi196 ceph-mon[67089]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:04:14.170 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:14 smithi132 ceph-mon[32020]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:04:14.170 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.170 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.171 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:14 smithi144 ceph-mon[66781]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:04:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:14.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:14 smithi196 ceph-mon[67089]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:04:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:16.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:16 smithi144 ceph-mon[66781]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:16.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:16 smithi132 ceph-mon[32020]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:16.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:16 smithi196 ceph-mon[67089]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:17.154 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 6 on host 'smithi132' 2024-03-18T23:04:17.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.153+0000 7f16daffd700 1 -- 172.21.15.132:0/858081764 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f16e4062810 con 0x7f16d00a2910 2024-03-18T23:04:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 msgr2=0x7f16d00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 0x7f16d00a4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16e406bb50 tx=0x7f16d400a380 comp rx=0 tx=0).stop 2024-03-18T23:04:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 msgr2=0x7f16e406b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bff0 tx=0x7f16e0005b00 comp rx=0 tx=0).stop 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 shutdown_connections 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f16d00a2910 0x7f16d00a4dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e4102df0 0x7f16e419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e4101250 0x7f16e406b5d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 --2- 172.21.15.132:0/858081764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e4102450 0x7f16e419d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 >> 172.21.15.132:0/858081764 conn(0x7f16e40fc810 msgr2=0x7f16e41056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 shutdown_connections 2024-03-18T23:04:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:04:17.157+0000 7f16ec07f700 1 -- 172.21.15.132:0/858081764 wait complete. 2024-03-18T23:04:17.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:17.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:17 smithi132 ceph-mon[32020]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:04:17.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:17.485 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:17.486 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:17 smithi196 ceph-mon[67089]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:17.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:17 smithi144 ceph-mon[66781]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T23:04:17.842 DEBUG:teuthology.orchestra.run.smithi132:osd.6> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.6.service 2024-03-18T23:04:17.845 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_2... 2024-03-18T23:04:17.845 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:04:18.090 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:04:19.262 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-mon[32020]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:04:19.262 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-mon[32020]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T23:04:19.262 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-mon[32020]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:04:19.262 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:19.263 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:04:19.122+0000 7f833b9f5700 -1 osd.6 0 waiting for initial osdmap 2024-03-18T23:04:19.263 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:04:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:04:19.131+0000 7f83377e8700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:04:19.383 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:19 smithi144 ceph-mon[66781]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:04:19.383 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:19 smithi144 ceph-mon[66781]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T23:04:19.383 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:19 smithi144 ceph-mon[66781]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:04:19.384 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:19 smithi196 ceph-mon[67089]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T23:04:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:19 smithi196 ceph-mon[67089]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T23:04:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:19 smithi196 ceph-mon[67089]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:04:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.041 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:04:20.042 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:04:20.042 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-18T23:04:20.042 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-18T23:04:20.042 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013008 s, 806 MB/s 2024-03-18T23:04:20.042 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-18T23:04:20.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:20 smithi144 ceph-mon[66781]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:20.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:20 smithi144 ceph-mon[66781]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:04:20.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:20 smithi144 ceph-mon[66781]: osdmap e47: 7 total, 6 up, 7 in 2024-03-18T23:04:20.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.331 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:20 smithi132 ceph-mon[32020]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:20 smithi132 ceph-mon[32020]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:04:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:20 smithi132 ceph-mon[32020]: osdmap e47: 7 total, 6 up, 7 in 2024-03-18T23:04:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:20 smithi196 ceph-mon[67089]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:20.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:20 smithi196 ceph-mon[67089]: from='osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:04:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:20 smithi196 ceph-mon[67089]: osdmap e47: 7 total, 6 up, 7 in 2024-03-18T23:04:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:20.625 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-03-18T23:04:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:04:21.259 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] boot 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:21.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:21.261 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] boot 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: osd.6 [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] boot 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:21.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:22.436 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:22 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi132 2024-03-18T23:04:22.436 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:22 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 5696M 2024-03-18T23:04:22.436 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:22 smithi132 ceph-mon[32020]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:22.436 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:22 smithi132 ceph-mon[32020]: osdmap e49: 7 total, 7 up, 7 in 2024-03-18T23:04:22.441 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:22 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi132 2024-03-18T23:04:22.441 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:22 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 5696M 2024-03-18T23:04:22.441 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:22 smithi144 ceph-mon[66781]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:22.441 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:22 smithi144 ceph-mon[66781]: osdmap e49: 7 total, 7 up, 7 in 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 -- 172.21.15.144:0/880776807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 msgr2=0x7f2764101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 --2- 172.21.15.144:0/880776807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f2764101650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f27600099c0 tx=0x7f2760009cd0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 -- 172.21.15.144:0/880776807 shutdown_connections 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 --2- 172.21.15.144:0/880776807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f2764101650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 --2- 172.21.15.144:0/880776807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102dd0 0x7f2764109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 --2- 172.21.15.144:0/880776807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102430 0x7f2764102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 -- 172.21.15.144:0/880776807 >> 172.21.15.144:0/880776807 conn(0x7f27640fc7d0 msgr2=0x7f27640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:22.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 -- 172.21.15.144:0/880776807 shutdown_connections 2024-03-18T23:04:22.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.452+0000 7f276bb91700 1 -- 172.21.15.144:0/880776807 wait complete. 2024-03-18T23:04:22.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 Processor -- start 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 -- start start 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102430 0x7f276419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102dd0 0x7f27641a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f276410cdf0 con 0x7f2764102430 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f276410cc70 con 0x7f2764101230 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276bb91700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f276410cf70 con 0x7f2764102dd0 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54086/0 (socket says 172.21.15.144:54086) 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 -- 172.21.15.144:0/3224480344 learned_addr learned my addr 172.21.15.144:0/3224480344 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276912c700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102430 0x7f276419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:22.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276a12e700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102dd0 0x7f27641a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 -- 172.21.15.144:0/3224480344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102dd0 msgr2=0x7f27641a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102dd0 0x7f27641a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 -- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102430 msgr2=0x7f276419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102430 0x7f276419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 -- 172.21.15.144:0/3224480344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27641a23b0 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.453+0000 7f276992d700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f276000b580 tx=0x7f2760005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.454+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760027070 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.454+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27641a2640 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.454+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2760004130 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.454+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760022de0 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.454+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641a2b70 con 0x7f2764101230 2024-03-18T23:04:22.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.455+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2764068fd0 con 0x7f2764101230 2024-03-18T23:04:22.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.456+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f276002a070 con 0x7f2764101230 2024-03-18T23:04:22.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.456+0000 7f275affd700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 0x7f27500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:22.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.457+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f27600cf170 con 0x7f2764101230 2024-03-18T23:04:22.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.457+0000 7f276912c700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 0x7f27500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:22.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.457+0000 7f276912c700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 0x7f27500a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2754005950 tx=0x7f275400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:22.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.460+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2760031070 con 0x7f2764101230 2024-03-18T23:04:22.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:22 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi132 2024-03-18T23:04:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:22 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 5696M 2024-03-18T23:04:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:22 smithi196 ceph-mon[67089]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-18T23:04:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:22 smithi196 ceph-mon[67089]: osdmap e49: 7 total, 7 up, 7 in 2024-03-18T23:04:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:22.665+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f27640fe9b0 con 0x7f27500a2890 2024-03-18T23:04:23.386 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:23.386 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:23.386 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:23.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:23.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:23.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:23.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:23.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:24.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:24 smithi144 ceph-mon[66781]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:24.322 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:24 smithi144 ceph-mon[66781]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:24.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:24 smithi196 ceph-mon[67089]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:24 smithi196 ceph-mon[67089]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:24 smithi132 ceph-mon[32020]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:24 smithi132 ceph-mon[32020]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/3246929367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]': finished 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:26 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/2863610234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/3246929367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]': finished 2024-03-18T23:04:26.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:04:26.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:26.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:26 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/2863610234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/3246929367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]: dispatch 2024-03-18T23:04:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf357f13-d06f-49c0-b598-1cadcd305fa3"}]': finished 2024-03-18T23:04:26.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T23:04:26.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:26.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:26 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/2863610234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:27.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:27 smithi196 ceph-mon[67089]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:27 smithi144 ceph-mon[66781]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:27.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:27 smithi132 ceph-mon[32020]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:30 smithi144 ceph-mon[66781]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:30.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:30 smithi132 ceph-mon[32020]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:30.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:30 smithi196 ceph-mon[67089]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:32 smithi144 ceph-mon[66781]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:32 smithi132 ceph-mon[32020]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:32.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:32 smithi196 ceph-mon[67089]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:33.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:33 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:04:33.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:33 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:33.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:04:33.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:33.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:33 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T23:04:33.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:33 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:34.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:34 smithi144 ceph-mon[66781]: Deploying daemon osd.7 on smithi144 2024-03-18T23:04:34.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:34 smithi144 ceph-mon[66781]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:34.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:34 smithi132 ceph-mon[32020]: Deploying daemon osd.7 on smithi144 2024-03-18T23:04:34.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:34 smithi132 ceph-mon[32020]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:34.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:34 smithi196 ceph-mon[67089]: Deploying daemon osd.7 on smithi144 2024-03-18T23:04:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:34 smithi196 ceph-mon[67089]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:36.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:36 smithi144 ceph-mon[66781]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:36 smithi132 ceph-mon[32020]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:36 smithi196 ceph-mon[67089]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:38 smithi144 ceph-mon[66781]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:38 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:38 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:38 smithi132 ceph-mon[32020]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:38 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:38 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:38.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:38 smithi196 ceph-mon[67089]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:38 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:38 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:39.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:39 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:39.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:39 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:39.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:39 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:40.345 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:40 smithi144 ceph-mon[66781]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:40.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:40 smithi132 ceph-mon[32020]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:40 smithi196 ceph-mon[67089]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.678+0000 7f275affd700 1 -- 172.21.15.144:0/3224480344 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f27640fe9b0 con 0x7f27500a2890 2024-03-18T23:04:41.678 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.680+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 msgr2=0x7f27500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.680+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 0x7f27500a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2754005950 tx=0x7f275400b410 comp rx=0 tx=0).stop 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.680+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 msgr2=0x7f276419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.680+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f276000b580 tx=0x7f2760005e80 comp rx=0 tx=0).stop 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 shutdown_connections 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f27500a2890 0x7f27500a4d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764101230 0x7f276419a7c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2764102dd0 0x7f27641a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:41.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 --2- 172.21.15.144:0/3224480344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764102430 0x7f276419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:41.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 >> 172.21.15.144:0/3224480344 conn(0x7f27640fc7d0 msgr2=0x7f27640fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:41.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 shutdown_connections 2024-03-18T23:04:41.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:04:41.681+0000 7f276bb91700 1 -- 172.21.15.144:0/3224480344 wait complete. 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:41 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.237 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:42.238 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.238 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.238 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:41 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:42.264 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.7.service 2024-03-18T23:04:42.267 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-03-18T23:04:42.267 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T23:04:42.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:42.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:41 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:04:42.513 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:04:43.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:42 smithi196 ceph-mon[67089]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:43.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:42 smithi196 ceph-mon[67089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:43.343 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:42 smithi144 ceph-mon[66781]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:43.343 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:42 smithi144 ceph-mon[66781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:43.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:42 smithi132 ceph-mon[32020]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:43.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:42 smithi132 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:43 smithi196 ceph-mon[67089]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:04:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T23:04:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:43 smithi144 ceph-mon[66781]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T23:04:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T23:04:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: from='osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:04:44.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:43 smithi132 ceph-mon[32020]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:44.385 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T23:04:44.386 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T23:04:44.386 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-18T23:04:44.386 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-18T23:04:44.386 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112075 s, 936 MB/s 2024-03-18T23:04:44.386 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-18T23:04:44.941 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:04:44.860+0000 7f9d4dd94700 -1 osd.7 0 waiting for initial osdmap 2024-03-18T23:04:44.982 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-03-18T23:04:45.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:44 smithi196 ceph-mon[67089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:04:45.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:44 smithi196 ceph-mon[67089]: osdmap e52: 8 total, 7 up, 8 in 2024-03-18T23:04:45.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:44 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:44 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:44 smithi196 ceph-mon[67089]: from='osd.7 ' entity='osd.7' 2024-03-18T23:04:45.231 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:04:45.325 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:04:44.996+0000 7f9d45b7f700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:04:45.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-mon[66781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:04:45.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-mon[66781]: osdmap e52: 8 total, 7 up, 8 in 2024-03-18T23:04:45.325 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.326 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.326 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:44 smithi144 ceph-mon[66781]: from='osd.7 ' entity='osd.7' 2024-03-18T23:04:45.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:44 smithi132 ceph-mon[32020]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:04:45.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:44 smithi132 ceph-mon[32020]: osdmap e52: 8 total, 7 up, 8 in 2024-03-18T23:04:45.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:44 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:45.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:44 smithi132 ceph-mon[32020]: from='osd.7 ' entity='osd.7' 2024-03-18T23:04:46.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:04:46.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:04:46.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: osdmap e53: 8 total, 7 up, 8 in 2024-03-18T23:04:46.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:46.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:46.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] boot 2024-03-18T23:04:46.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:04:46.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:04:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:04:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: osdmap e53: 8 total, 7 up, 8 in 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:46.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:46.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] boot 2024-03-18T23:04:46.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:04:46.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: osdmap e53: 8 total, 7 up, 8 in 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:04:46.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: osd.7 [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] boot 2024-03-18T23:04:46.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T23:04:46.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:04:46.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.814+0000 7f181b30b700 1 -- 172.21.15.196:0/3358050056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 msgr2=0x7f181410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:46.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.814+0000 7f181b30b700 1 --2- 172.21.15.196:0/3358050056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181410a4a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f180c00b0a0 tx=0x7f180c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- 172.21.15.196:0/3358050056 shutdown_connections 2024-03-18T23:04:46.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- 172.21.15.196:0/3358050056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- 172.21.15.196:0/3358050056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181410a4a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- 172.21.15.196:0/3358050056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f1814107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- 172.21.15.196:0/3358050056 >> 172.21.15.196:0/3358050056 conn(0x7f18140fc800 msgr2=0x7f18140fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:04:46.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- 172.21.15.196:0/3358050056 shutdown_connections 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- 172.21.15.196:0/3358050056 wait complete. 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 Processor -- start 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- start start 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f18141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:46.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:46.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.815+0000 7f181b30b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f181410def0 con 0x7f1814105700 2024-03-18T23:04:46.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f181b30b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f181410dd70 con 0x7f1814102dd0 2024-03-18T23:04:46.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f181b30b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f181410e070 con 0x7f1814108030 2024-03-18T23:04:46.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:46.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43208/0 (socket says 172.21.15.196:43208) 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 -- 172.21.15.196:0/4198064575 learned_addr learned my addr 172.21.15.196:0/4198064575 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18188a6700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f18141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18190a7700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 -- 172.21.15.196:0/4198064575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 msgr2=0x7f18141984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 -- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 msgr2=0x7f18141989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:04:46.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f18141989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:04:46.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 -- 172.21.15.196:0/4198064575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18141a0110 con 0x7f1814108030 2024-03-18T23:04:46.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18190a7700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:04:46.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18188a6700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f18141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:04:46.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f18198a8700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f180c00b070 tx=0x7f180c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c021070 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c007b80 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141a03a0 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c01d450 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.816+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a08a0 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.817+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1814068fd0 con 0x7f1814108030 2024-03-18T23:04:46.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.818+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f180c004000 con 0x7f1814108030 2024-03-18T23:04:46.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.819+0000 7f180a7fc700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:04:46.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.819+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5430+0+0 (secure 0 0 0) 0x7f180c00f020 con 0x7f1814108030 2024-03-18T23:04:46.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.819+0000 7f18190a7700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:04:46.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.820+0000 7f18190a7700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 0x7f18000a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1810005fd0 tx=0x7f1810005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:04:46.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:46.823+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f180c02a020 con 0x7f1814108030 2024-03-18T23:04:47.028 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:04:47.028+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1814062750 con 0x7f18000a2910 2024-03-18T23:04:47.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:46 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi144 2024-03-18T23:04:47.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:46 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-18T23:04:47.046 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:46 smithi196 ceph-mon[67089]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:04:47.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:46 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi144 2024-03-18T23:04:47.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:46 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-18T23:04:47.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:46 smithi144 ceph-mon[66781]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:04:47.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:46 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi144 2024-03-18T23:04:47.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:46 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-18T23:04:47.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:46 smithi132 ceph-mon[32020]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:47 smithi196 ceph-mon[67089]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:47 smithi144 ceph-mon[66781]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T23:04:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:04:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:04:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:04:48.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:48.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:47 smithi132 ceph-mon[32020]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T23:04:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:49 smithi196 ceph-mon[67089]: osdmap e57: 8 total, 8 up, 8 in 2024-03-18T23:04:49.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:49 smithi196 ceph-mon[67089]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:50.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:49 smithi144 ceph-mon[66781]: osdmap e57: 8 total, 8 up, 8 in 2024-03-18T23:04:50.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:49 smithi144 ceph-mon[66781]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:50.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:49 smithi132 ceph-mon[32020]: osdmap e57: 8 total, 8 up, 8 in 2024-03-18T23:04:50.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:49 smithi132 ceph-mon[32020]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: osdmap e58: 8 total, 8 up, 8 in 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/2808016644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]': finished 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:04:50.987 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:50.988 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:50 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/858681953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: osdmap e58: 8 total, 8 up, 8 in 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/2808016644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]': finished 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:51.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:50 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/858681953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: osdmap e58: 8 total, 8 up, 8 in 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/2808016644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]: dispatch 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8d245b3-4a54-4056-92f6-0c30db4759cb"}]': finished 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:04:51.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:50 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/858681953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:04:51.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:51 smithi132 ceph-mon[32020]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:52.010 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:51 smithi144 ceph-mon[66781]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:52.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:51 smithi196 ceph-mon[67089]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:54.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:54 smithi196 ceph-mon[67089]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:54.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:54 smithi144 ceph-mon[66781]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:54.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:54 smithi132 ceph-mon[32020]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:56.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:56 smithi196 ceph-mon[67089]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:56.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:56 smithi144 ceph-mon[66781]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:56 smithi132 ceph-mon[32020]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:57.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:04:57.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:04:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T23:04:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:04:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:58 smithi196 ceph-mon[67089]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:04:58 smithi196 ceph-mon[67089]: Deploying daemon osd.8 on smithi196 2024-03-18T23:04:58.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:58 smithi144 ceph-mon[66781]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:58.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:04:58 smithi144 ceph-mon[66781]: Deploying daemon osd.8 on smithi196 2024-03-18T23:04:58.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:58 smithi132 ceph-mon[32020]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:04:58.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:04:58 smithi132 ceph-mon[32020]: Deploying daemon osd.8 on smithi196 2024-03-18T23:05:00.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:00 smithi144 ceph-mon[66781]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:00 smithi132 ceph-mon[32020]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:00.411 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:00 smithi196 ceph-mon[67089]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:02.292 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:02 smithi196 ceph-mon[67089]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:02 smithi144 ceph-mon[66781]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:02 smithi132 ceph-mon[32020]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:03 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:03 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:03 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:03 smithi144 ceph-mon[66781]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:03.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:03 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:03 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:03 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:03.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:03 smithi196 ceph-mon[67089]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:03.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:03 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:03 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:03.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:03 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:03.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:03 smithi132 ceph-mon[32020]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:05.742 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-03-18T23:05:05.742 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.741+0000 7f180a7fc700 1 -- 172.21.15.196:0/4198064575 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1814062750 con 0x7f18000a2910 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 msgr2=0x7f18000a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 0x7f18000a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1810005fd0 tx=0x7f1810005f60 comp rx=0 tx=0).stop 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 msgr2=0x7f181419fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f180c00b070 tx=0x7f180c004d00 comp rx=0 tx=0).stop 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 shutdown_connections 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f18000a2910 0x7f18000a4dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:05.744 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1814102dd0 0x7f18141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:05.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1814108030 0x7f181419fa70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:05.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 --2- 172.21.15.196:0/4198064575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1814105700 0x7f18141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:05.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 >> 172.21.15.196:0/4198064575 conn(0x7f18140fc800 msgr2=0x7f1814103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:05.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 shutdown_connections 2024-03-18T23:05:05.745 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:05:05.743+0000 7f181b30b700 1 -- 172.21.15.196:0/4198064575 wait complete. 2024-03-18T23:05:06.350 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:06 smithi196 ceph-mon[67089]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:06.350 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:06 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.351 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:06 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:06 smithi144 ceph-mon[66781]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:06 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:06 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:06 smithi132 ceph-mon[32020]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:06.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:06 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:06 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:06.376 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.8.service 2024-03-18T23:05:06.378 INFO:tasks.cephadm:Deploying osd.9 on smithi132 with /dev/vg_nvme/lv_1... 2024-03-18T23:05:06.378 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:05:06.631 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:05:07.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:07 smithi144 ceph-mon[66781]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:07 smithi144 ceph-mon[66781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:07.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:07.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:07 smithi132 ceph-mon[32020]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:07 smithi132 ceph-mon[32020]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:07.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:07.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:07 smithi196 ceph-mon[67089]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:07 smithi196 ceph-mon[67089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T23:05:07.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:07.404 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: osdmap e60: 9 total, 8 up, 9 in 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-mon[67089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.225 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:05:08.083+0000 7ffaed418700 -1 osd.8 0 waiting for initial osdmap 2024-03-18T23:05:08.225 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:05:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:05:08.098+0000 7ffae920b700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: osdmap e60: 9 total, 8 up, 9 in 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:08 smithi144 ceph-mon[66781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:08.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T23:05:08.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: osdmap e60: 9 total, 8 up, 9 in 2024-03-18T23:05:08.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:08.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: from='osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:08 smithi132 ceph-mon[32020]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:05:08.630 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:05:08.631 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:05:08.631 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-03-18T23:05:08.631 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-03-18T23:05:08.631 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114755 s, 914 MB/s 2024-03-18T23:05:08.631 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-03-18T23:05:09.350 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: osdmap e61: 9 total, 8 up, 9 in 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] boot 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:05:09.351 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: osdmap e61: 9 total, 8 up, 9 in 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] boot 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:05:09.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.378 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: osdmap e61: 9 total, 8 up, 9 in 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: osd.8 [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] boot 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T23:05:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:05:09.631 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:05:10.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:05:10.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.368 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:05:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:05:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:10.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:11.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- 172.21.15.132:0/2940576228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 msgr2=0x7f72a8109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:11.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- 172.21.15.132:0/2940576228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a8109690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f72a000b0d0 tx=0x7f72a000b3e0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- 172.21.15.132:0/2940576228 shutdown_connections 2024-03-18T23:05:11.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- 172.21.15.132:0/2940576228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8101240 0x7f72a8101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- 172.21.15.132:0/2940576228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a8109690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- 172.21.15.132:0/2940576228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8102440 0x7f72a81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- 172.21.15.132:0/2940576228 >> 172.21.15.132:0/2940576228 conn(0x7f72a80fc800 msgr2=0x7f72a80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- 172.21.15.132:0/2940576228 shutdown_connections 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- 172.21.15.132:0/2940576228 wait complete. 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 Processor -- start 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- start start 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8101240 0x7f72a8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 0x7f72a8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810cec0 con 0x7f72a8102de0 2024-03-18T23:05:11.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810cd40 con 0x7f72a8102440 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72aef87700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72a810d040 con 0x7f72a8101240 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.263+0000 7f72ad524700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47862/0 (socket says 172.21.15.132:47862) 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72acd23700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8101240 0x7f72a8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 -- 172.21.15.132:0/2284460206 learned_addr learned my addr 172.21.15.132:0/2284460206 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72a7fff700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 0x7f72a8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 -- 172.21.15.132:0/2284460206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8101240 msgr2=0x7f72a8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8101240 0x7f72a8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 -- 172.21.15.132:0/2284460206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 msgr2=0x7f72a8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 0x7f72a8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 -- 172.21.15.132:0/2284460206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a81a01d0 con 0x7f72a8102de0 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72a7fff700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 0x7f72a8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72ad524700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007d10 tx=0x7f72a0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:11.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0021020 con 0x7f72a8102de0 2024-03-18T23:05:11.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81a0460 con 0x7f72a8102de0 2024-03-18T23:05:11.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a001d950 con 0x7f72a8102de0 2024-03-18T23:05:11.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0005450 con 0x7f72a8102de0 2024-03-18T23:05:11.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.264+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81a0a70 con 0x7f72a8102de0 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.265+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a8068fd0 con 0x7f72a8102de0 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.266+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f72a0024020 con 0x7f72a8102de0 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.267+0000 7f72a5ffb700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 0x7f72980a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.267+0000 7f72acd23700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 0x7f72980a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.267+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5858+0+0 (secure 0 0 0) 0x7f72a000f020 con 0x7f72a8102de0 2024-03-18T23:05:11.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.268+0000 7f72acd23700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 0x7f72980a4d50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f72a81022a0 tx=0x7f729c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:11.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.271+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f72a009d580 con 0x7f72a8102de0 2024-03-18T23:05:11.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:11 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi196 2024-03-18T23:05:11.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:11 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 5693M 2024-03-18T23:05:11.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:11 smithi144 ceph-mon[66781]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:05:11.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:11 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi196 2024-03-18T23:05:11.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:11 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 5693M 2024-03-18T23:05:11.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:11 smithi132 ceph-mon[32020]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:05:11.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:11 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi196 2024-03-18T23:05:11.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:11 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 5693M 2024-03-18T23:05:11.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:11 smithi196 ceph-mon[67089]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T23:05:11.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:11.481+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f72a81a0d90 con 0x7f72980a2890 2024-03-18T23:05:12.355 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:12 smithi132 ceph-mon[32020]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:12.356 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:12.356 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:12.356 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:12.356 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:12 smithi132 ceph-mon[32020]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:05:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:12 smithi144 ceph-mon[66781]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:12.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:12 smithi144 ceph-mon[66781]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:05:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:12 smithi196 ceph-mon[67089]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:12 smithi196 ceph-mon[67089]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-18T23:05:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:13 smithi144 ceph-mon[66781]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:13.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:13 smithi144 ceph-mon[66781]: osdmap e65: 9 total, 9 up, 9 in 2024-03-18T23:05:13.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:13 smithi132 ceph-mon[32020]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:13.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:13 smithi132 ceph-mon[32020]: osdmap e65: 9 total, 9 up, 9 in 2024-03-18T23:05:13.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:13 smithi196 ceph-mon[67089]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:13.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:13 smithi196 ceph-mon[67089]: osdmap e65: 9 total, 9 up, 9 in 2024-03-18T23:05:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:14 smithi144 ceph-mon[66781]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:14 smithi144 ceph-mon[66781]: osdmap e66: 9 total, 9 up, 9 in 2024-03-18T23:05:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:14 smithi132 ceph-mon[32020]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:14.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:14 smithi132 ceph-mon[32020]: osdmap e66: 9 total, 9 up, 9 in 2024-03-18T23:05:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:14 smithi196 ceph-mon[67089]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:14 smithi196 ceph-mon[67089]: osdmap e66: 9 total, 9 up, 9 in 2024-03-18T23:05:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]: dispatch 2024-03-18T23:05:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]': finished 2024-03-18T23:05:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:15 smithi196 ceph-mon[67089]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:05:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:15 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4240755939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]: dispatch 2024-03-18T23:05:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]': finished 2024-03-18T23:05:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:15 smithi144 ceph-mon[66781]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:05:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:15 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4240755939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]: dispatch 2024-03-18T23:05:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3859170232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b2f5b52-8981-48f2-9229-27493b6d62f2"}]': finished 2024-03-18T23:05:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:15 smithi132 ceph-mon[32020]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T23:05:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:15 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4240755939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:16.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:16 smithi196 ceph-mon[67089]: pgmap v205: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:16.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:16 smithi144 ceph-mon[66781]: pgmap v205: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:16.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:16 smithi132 ceph-mon[32020]: pgmap v205: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-18T23:05:17.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:17 smithi196 ceph-mon[67089]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:05:17.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:17 smithi196 ceph-mon[67089]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:05:17.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:17 smithi196 ceph-mon[67089]: Cluster is now healthy 2024-03-18T23:05:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:17 smithi144 ceph-mon[66781]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:05:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:17 smithi144 ceph-mon[66781]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:05:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:17 smithi144 ceph-mon[66781]: Cluster is now healthy 2024-03-18T23:05:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:17 smithi132 ceph-mon[32020]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:05:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:17 smithi132 ceph-mon[32020]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-18T23:05:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:17 smithi132 ceph-mon[32020]: Cluster is now healthy 2024-03-18T23:05:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:20 smithi144 ceph-mon[66781]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-18T23:05:20.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:20 smithi132 ceph-mon[32020]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-18T23:05:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:20 smithi196 ceph-mon[67089]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-18T23:05:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:22 smithi144 ceph-mon[66781]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:05:22.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:22 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:22.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:22 smithi132 ceph-mon[32020]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:22.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:05:22.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:22 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:22 smithi196 ceph-mon[67089]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:05:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T23:05:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:22 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:23.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:23 smithi144 ceph-mon[66781]: Deploying daemon osd.9 on smithi132 2024-03-18T23:05:23.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:23 smithi132 ceph-mon[32020]: Deploying daemon osd.9 on smithi132 2024-03-18T23:05:23.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:23 smithi196 ceph-mon[67089]: Deploying daemon osd.9 on smithi132 2024-03-18T23:05:24.082 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:24 smithi132 ceph-mon[32020]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:05:24.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:24 smithi144 ceph-mon[66781]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:05:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:24 smithi196 ceph-mon[67089]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:05:26.321 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:26 smithi132 ceph-mon[32020]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:05:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:26 smithi144 ceph-mon[66781]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:05:26.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:26 smithi196 ceph-mon[67089]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T23:05:28.201 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:28 smithi132 ceph-mon[32020]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:05:28.202 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.202 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.202 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:28 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:28 smithi144 ceph-mon[66781]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:05:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:28 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:28 smithi196 ceph-mon[67089]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:05:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:28 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:30 smithi144 ceph-mon[66781]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:30.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:30 smithi132 ceph-mon[32020]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:30.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:30 smithi196 ceph-mon[67089]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:31.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.240+0000 7f72a5ffb700 1 -- 172.21.15.132:0/2284460206 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72a81a0d90 con 0x7f72980a2890 2024-03-18T23:05:31.241 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 9 on host 'smithi132' 2024-03-18T23:05:31.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.242+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 msgr2=0x7f72980a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.242+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 0x7f72980a4d50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f72a81022a0 tx=0x7f729c009190 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.242+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 msgr2=0x7f72a819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.242+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f72a0007d10 tx=0x7f72a0007d40 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.242+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 shutdown_connections 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72980a2890 0x7f72980a4d50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72a8102de0 0x7f72a819fb30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72a8101240 0x7f72a8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 --2- 172.21.15.132:0/2284460206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72a8102440 0x7f72a8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 >> 172.21.15.132:0/2284460206 conn(0x7f72a80fc800 msgr2=0x7f72a80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 shutdown_connections 2024-03-18T23:05:31.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:05:31.243+0000 7f72aef87700 1 -- 172.21.15.132:0/2284460206 wait complete. 2024-03-18T23:05:31.429 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.430 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:31 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:31.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:31 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:31.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:31 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:31.974 DEBUG:teuthology.orchestra.run.smithi132:osd.9> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.9.service 2024-03-18T23:05:31.976 INFO:tasks.cephadm:Deploying osd.10 on smithi144 with /dev/vg_nvme/lv_1... 2024-03-18T23:05:31.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:05:32.224 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:05:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:32 smithi144 ceph-mon[66781]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:32 smithi144 ceph-mon[66781]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:32.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:32 smithi132 ceph-mon[32020]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:32.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:32 smithi132 ceph-mon[32020]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:32.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:32 smithi196 ceph-mon[67089]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:32.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:32 smithi196 ceph-mon[67089]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: osdmap e68: 10 total, 9 up, 10 in 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:33 smithi132 ceph-mon[32020]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: osdmap e68: 10 total, 9 up, 10 in 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.647 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:33 smithi144 ceph-mon[66781]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: osdmap e68: 10 total, 9 up, 10 in 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: from='osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-03-18T23:05:33.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:33 smithi196 ceph-mon[67089]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113088 s, 927 MB/s 2024-03-18T23:05:34.218 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-18T23:05:34.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-mon[32020]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:05:34.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-mon[32020]: osdmap e69: 10 total, 9 up, 10 in 2024-03-18T23:05:34.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-mon[32020]: from='osd.9 ' entity='osd.9' 2024-03-18T23:05:34.494 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:05:34.326+0000 7fc29a53b700 -1 osd.9 0 waiting for initial osdmap 2024-03-18T23:05:34.494 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:05:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:05:34.492+0000 7fc292326700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:05:34.684 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:34 smithi144 ceph-mon[66781]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:05:34.684 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:34 smithi144 ceph-mon[66781]: osdmap e69: 10 total, 9 up, 10 in 2024-03-18T23:05:34.684 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.684 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:34 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.684 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:34 smithi144 ceph-mon[66781]: from='osd.9 ' entity='osd.9' 2024-03-18T23:05:34.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:34 smithi196 ceph-mon[67089]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-03-18T23:05:34.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:34 smithi196 ceph-mon[67089]: osdmap e69: 10 total, 9 up, 10 in 2024-03-18T23:05:34.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:34 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:34.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:34 smithi196 ceph-mon[67089]: from='osd.9 ' entity='osd.9' 2024-03-18T23:05:34.805 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-03-18T23:05:35.052 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:05:35.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:05:35.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:05:35.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:35.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi132 2024-03-18T23:05:35.464 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 4272M 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.465 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:35 smithi144 ceph-mon[66781]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi132 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 4272M 2024-03-18T23:05:35.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:35.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:35.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:35 smithi196 ceph-mon[67089]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:35.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi132 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 4272M 2024-03-18T23:05:35.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:35.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:05:35.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:35.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:35 smithi132 ceph-mon[32020]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-18T23:05:36.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1485491935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24102340 msgr2=0x7f6f24102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:36.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1485491935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24102340 0x7f6f24102760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f140099c0 tx=0x7f6f14009cd0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1485491935 shutdown_connections 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1485491935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24102340 0x7f6f24102760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1485491935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103ee0 0x7f6f2410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1485491935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24103540 0x7f6f241039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1485491935 >> 172.21.15.144:0/1485491935 conn(0x7f6f240fd920 msgr2=0x7f6f240ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1485491935 shutdown_connections 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1485491935 wait complete. 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.565+0000 7f6f2a45b700 1 Processor -- start 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 -- start start 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24102340 0x7f6f24198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103540 0x7f6f24198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:36.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410df90 con 0x7f6f24102340 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410de10 con 0x7f6f24103ee0 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f2a45b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2410e110 con 0x7f6f24103540 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50202/0 (socket says 172.21.15.144:50202) 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 -- 172.21.15.144:0/1256973391 learned_addr learned my addr 172.21.15.144:0/1256973391 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f237fe700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103540 0x7f6f24198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f23fff700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24102340 0x7f6f24198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 -- 172.21.15.144:0/1256973391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103540 msgr2=0x7f6f24198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103540 0x7f6f24198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 -- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24102340 msgr2=0x7f6f24198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24102340 0x7f6f24198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 -- 172.21.15.144:0/1256973391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f241a01a0 con 0x7f6f24103ee0 2024-03-18T23:05:36.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.566+0000 7f6f289f8700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b970 tx=0x7f6f1c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.567+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00d860 con 0x7f6f24103ee0 2024-03-18T23:05:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.567+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f1c00e430 con 0x7f6f24103ee0 2024-03-18T23:05:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.567+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f241a0490 con 0x7f6f24103ee0 2024-03-18T23:05:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.567+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c017660 con 0x7f6f24103ee0 2024-03-18T23:05:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.567+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f241a0a00 con 0x7f6f24103ee0 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.568+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f24068fd0 con 0x7f6f24103ee0 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.569+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f6f1c00e950 con 0x7f6f24103ee0 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.569+0000 7f6f217fa700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 0x7f6f0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.570+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7f6f1c0cddf0 con 0x7f6f24103ee0 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.570+0000 7f6f23fff700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 0x7f6f0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:05:36.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.570+0000 7f6f23fff700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 0x7f6f0c0a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f14003890 tx=0x7f6f14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:05:36.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.574+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6f1c013030 con 0x7f6f24103ee0 2024-03-18T23:05:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:36 smithi144 ceph-mon[66781]: osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] boot 2024-03-18T23:05:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:36 smithi144 ceph-mon[66781]: osdmap e70: 10 total, 10 up, 10 in 2024-03-18T23:05:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:36.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:36.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:36 smithi196 ceph-mon[67089]: osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] boot 2024-03-18T23:05:36.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:36 smithi196 ceph-mon[67089]: osdmap e70: 10 total, 10 up, 10 in 2024-03-18T23:05:36.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:36.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:36.783+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6f24062750 con 0x7f6f0c0a2890 2024-03-18T23:05:36.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:36.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:36 smithi132 ceph-mon[32020]: osd.9 [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] boot 2024-03-18T23:05:36.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:36 smithi132 ceph-mon[32020]: osdmap e70: 10 total, 10 up, 10 in 2024-03-18T23:05:36.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: osdmap e71: 10 total, 10 up, 10 in 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:37.666 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:37 smithi144 ceph-mon[66781]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: osdmap e71: 10 total, 10 up, 10 in 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:37.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:37 smithi196 ceph-mon[67089]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:37.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: osdmap e71: 10 total, 10 up, 10 in 2024-03-18T23:05:37.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:05:37.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:05:37.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:05:37.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:37.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:37 smithi132 ceph-mon[32020]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/767901939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]': finished 2024-03-18T23:05:40.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:05:40.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:40.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.144:0/2941349875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:40.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:40.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/767901939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]': finished 2024-03-18T23:05:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:05:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.144:0/2941349875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/767901939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]: dispatch 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5922d4cd-937c-44ec-b82c-7fec2f16c8c4"}]': finished 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.144:0/2941349875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:05:42.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:42 smithi144 ceph-mon[66781]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:42.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:42 smithi132 ceph-mon[32020]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:42 smithi196 ceph-mon[67089]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:44 smithi144 ceph-mon[66781]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:44 smithi132 ceph-mon[32020]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:44.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:44 smithi196 ceph-mon[67089]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:46.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:46 smithi144 ceph-mon[66781]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:46.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:46 smithi132 ceph-mon[32020]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:46.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:46 smithi196 ceph-mon[67089]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:47.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:05:47.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:47 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:47.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:05:47.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:47 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:47.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T23:05:47.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:47 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:05:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:48 smithi144 ceph-mon[66781]: Deploying daemon osd.10 on smithi144 2024-03-18T23:05:48.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:48 smithi144 ceph-mon[66781]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:48 smithi132 ceph-mon[32020]: Deploying daemon osd.10 on smithi144 2024-03-18T23:05:48.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:48 smithi132 ceph-mon[32020]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:48.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:48 smithi196 ceph-mon[67089]: Deploying daemon osd.10 on smithi144 2024-03-18T23:05:48.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:48 smithi196 ceph-mon[67089]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:50.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:50 smithi144 ceph-mon[66781]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:50 smithi132 ceph-mon[32020]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:50.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:50 smithi196 ceph-mon[67089]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:52.151 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:52 smithi144 ceph-mon[66781]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:52.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:52 smithi132 ceph-mon[32020]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:52.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:52 smithi196 ceph-mon[67089]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:53 smithi144 ceph-mon[66781]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:53 smithi132 ceph-mon[32020]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:05:53.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:53 smithi196 ceph-mon[67089]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:55.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.345+0000 7f6f217fa700 1 -- 172.21.15.144:0/1256973391 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6f24062750 con 0x7f6f0c0a2890 2024-03-18T23:05:55.345 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 10 on host 'smithi144' 2024-03-18T23:05:55.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 msgr2=0x7f6f0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:55.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 0x7f6f0c0a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f14003890 tx=0x7f6f14005fb0 comp rx=0 tx=0).stop 2024-03-18T23:05:55.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 msgr2=0x7f6f2419fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:05:55.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b970 tx=0x7f6f1c00bc80 comp rx=0 tx=0).stop 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 shutdown_connections 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6f0c0a2890 0x7f6f0c0a4d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f24103ee0 0x7f6f2419fb00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f24102340 0x7f6f24198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 --2- 172.21.15.144:0/1256973391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f24103540 0x7f6f24198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 >> 172.21.15.144:0/1256973391 conn(0x7f6f240fd920 msgr2=0x7f6f241068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 shutdown_connections 2024-03-18T23:05:55.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-18T23:05:55.347+0000 7f6f2a45b700 1 -- 172.21.15.144:0/1256973391 wait complete. 2024-03-18T23:05:56.172 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:56 smithi144 ceph-mon[66781]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:56.173 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.173 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.234 DEBUG:teuthology.orchestra.run.smithi144:osd.10> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.10.service 2024-03-18T23:05:56.236 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-03-18T23:05:56.236 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T23:05:56.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:56 smithi132 ceph-mon[32020]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:56.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:56 smithi196 ceph-mon[67089]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:56.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:56.483 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:05:57.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:57 smithi196 ceph-mon[67089]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:57 smithi196 ceph-mon[67089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:57.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:57 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:57 smithi144 ceph-mon[66781]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:57 smithi144 ceph-mon[66781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:57.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:57 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:57 smithi132 ceph-mon[32020]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:57 smithi132 ceph-mon[32020]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:57.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:57 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: osdmap e73: 11 total, 10 up, 11 in 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:58 smithi196 ceph-mon[67089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:58.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:05:58.293 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: osdmap e73: 11 total, 10 up, 11 in 2024-03-18T23:05:58.293 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:58.293 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.293 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:58 smithi144 ceph-mon[66781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:05:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T23:05:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: osdmap e73: 11 total, 10 up, 11 in 2024-03-18T23:05:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: from='osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:58 smithi132 ceph-mon[32020]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0172819 s, 607 MB/s 2024-03-18T23:05:58.514 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: osdmap e74: 11 total, 10 up, 11 in 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: osdmap e75: 11 total, 10 up, 11 in 2024-03-18T23:05:59.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.278 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-mon[66781]: from='osd.10 ' entity='osd.10' 2024-03-18T23:05:59.278 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:05:59.019+0000 7faa6f145700 -1 osd.10 0 waiting for initial osdmap 2024-03-18T23:05:59.278 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:05:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:05:59.202+0000 7faa66f30700 -1 osd.10 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:05:59.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: osdmap e74: 11 total, 10 up, 11 in 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: osdmap e75: 11 total, 10 up, 11 in 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:05:59 smithi132 ceph-mon[32020]: from='osd.10 ' entity='osd.10' 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: osdmap e74: 11 total, 10 up, 11 in 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: osdmap e75: 11 total, 10 up, 11 in 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:05:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:05:59 smithi196 ceph-mon[67089]: from='osd.10 ' entity='osd.10' 2024-03-18T23:05:59.544 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] boot 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:06:00.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:00.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.226 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:00 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:06:00.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:06:00.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] boot 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:00 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: osd.10 [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] boot 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:00.367 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:00 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/913088982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 msgr2=0x7fdc2410a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/913088982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2410a780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b0a0 tx=0x7fdc1c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/913088982 shutdown_connections 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/913088982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2410a780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/913088982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 0x7fdc24102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/913088982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 0x7fdc24103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.148 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/913088982 >> 172.21.15.196:0/913088982 conn(0x7fdc240fd8f0 msgr2=0x7fdc240ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/913088982 shutdown_connections 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.145+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/913088982 wait complete. 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 Processor -- start 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 -- start start 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 0x7fdc24198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 0x7fdc24198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410e010 con 0x7fdc24102330 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410de90 con 0x7fdc24103530 2024-03-18T23:06:01.149 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc2ab5e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410e190 con 0x7fdc24103ed0 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:44138/0 (socket says 172.21.15.196:44138) 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 -- 172.21.15.196:0/3990820013 learned_addr learned my addr 172.21.15.196:0/3990820013 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc23fff700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 0x7fdc24198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc288fa700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 0x7fdc24198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 -- 172.21.15.196:0/3990820013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 msgr2=0x7fdc24198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 0x7fdc24198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 -- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 msgr2=0x7fdc24198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 0x7fdc24198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:01.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 -- 172.21.15.196:0/3990820013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241a02b0 con 0x7fdc24103ed0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.146+0000 7fdc290fb700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00be60 tx=0x7fdc1c00be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.147+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c021020 con 0x7fdc24103ed0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.147+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc241a0540 con 0x7fdc24103ed0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.147+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc1c005520 con 0x7fdc24103ed0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.147+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c01b970 con 0x7fdc24103ed0 2024-03-18T23:06:01.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.147+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a0a70 con 0x7fdc24103ed0 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.148+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24068fd0 con 0x7fdc24103ed0 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.149+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fdc1c024020 con 0x7fdc24103ed0 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.149+0000 7fdc21ffb700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 0x7fdc140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.150+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7fdc1c00f020 con 0x7fdc24103ed0 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.150+0000 7fdc288fa700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 0x7fdc140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:01.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.150+0000 7fdc288fa700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 0x7fdc140a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc18006fd0 tx=0x7fdc18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:01.153 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.153+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc1c09d1d0 con 0x7fdc24103ed0 2024-03-18T23:06:01.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:01 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi144 2024-03-18T23:06:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:01 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-18T23:06:01.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:01 smithi196 ceph-mon[67089]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:06:01.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:01.357+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdc24062750 con 0x7fdc140a2890 2024-03-18T23:06:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:01 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi144 2024-03-18T23:06:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:01 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-18T23:06:01.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:01 smithi144 ceph-mon[66781]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:06:01.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:01 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi144 2024-03-18T23:06:01.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:01 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-18T23:06:01.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:01 smithi132 ceph-mon[32020]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T23:06:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:02 smithi196 ceph-mon[67089]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:06:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:06:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:02 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:02 smithi196 ceph-mon[67089]: osdmap e78: 11 total, 11 up, 11 in 2024-03-18T23:06:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:02 smithi144 ceph-mon[66781]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:06:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:06:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:02 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:02 smithi144 ceph-mon[66781]: osdmap e78: 11 total, 11 up, 11 in 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:02 smithi132 ceph-mon[32020]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T23:06:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:02 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:02 smithi132 ceph-mon[32020]: osdmap e78: 11 total, 11 up, 11 in 2024-03-18T23:06:03.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:03 smithi144 ceph-mon[66781]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:03.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:03 smithi132 ceph-mon[32020]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:03.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:03 smithi196 ceph-mon[67089]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:04.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:04.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: osdmap e79: 11 total, 11 up, 11 in 2024-03-18T23:06:04.362 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/3304592251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.363 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.363 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]': finished 2024-03-18T23:06:04.363 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:06:04.363 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: osdmap e79: 11 total, 11 up, 11 in 2024-03-18T23:06:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/3304592251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]': finished 2024-03-18T23:06:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:06:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 724 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: osdmap e79: 11 total, 11 up, 11 in 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/3304592251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]: dispatch 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d26113fc-541e-4090-89e8-909b4c0d5d05"}]': finished 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T23:06:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:05.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:05 smithi132 ceph-mon[32020]: from='client.? 172.21.15.196:0/350372828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:06:05.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:05 smithi196 ceph-mon[67089]: from='client.? 172.21.15.196:0/350372828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:06:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:05 smithi144 ceph-mon[66781]: from='client.? 172.21.15.196:0/350372828' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T23:06:06.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:06 smithi132 ceph-mon[32020]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 725 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:06 smithi196 ceph-mon[67089]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 725 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:06.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:06 smithi144 ceph-mon[66781]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 725 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:08.197 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:08 smithi196 ceph-mon[67089]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:06:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:08 smithi144 ceph-mon[66781]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:06:08.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:08 smithi132 ceph-mon[32020]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T23:06:10.418 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:10 smithi132 ceph-mon[32020]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T23:06:10.438 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:10 smithi196 ceph-mon[67089]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T23:06:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:10 smithi144 ceph-mon[66781]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T23:06:12.374 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:12 smithi196 ceph-mon[67089]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:06:12.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:06:12.375 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:12 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:12 smithi144 ceph-mon[66781]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:06:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:06:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:12 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:12.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:12 smithi132 ceph-mon[32020]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T23:06:12.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T23:06:12.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:12 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:13.437 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:13 smithi196 ceph-mon[67089]: Deploying daemon osd.11 on smithi196 2024-03-18T23:06:13.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:13 smithi144 ceph-mon[66781]: Deploying daemon osd.11 on smithi196 2024-03-18T23:06:13.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:13 smithi132 ceph-mon[32020]: Deploying daemon osd.11 on smithi196 2024-03-18T23:06:14.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:14 smithi196 ceph-mon[67089]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:06:14.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:14 smithi144 ceph-mon[66781]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:06:14.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:14 smithi132 ceph-mon[32020]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T23:06:16.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:16 smithi196 ceph-mon[67089]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:06:16.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:16 smithi144 ceph-mon[66781]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:06:16.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:16 smithi132 ceph-mon[32020]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-18T23:06:18.262 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:18 smithi196 ceph-mon[67089]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:06:18.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:18 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:18 smithi144 ceph-mon[66781]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:06:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:18 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:18 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:18 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:18 smithi132 ceph-mon[32020]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-18T23:06:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:18 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:20.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:20 smithi196 ceph-mon[67089]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:20 smithi144 ceph-mon[66781]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:20.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:20 smithi132 ceph-mon[32020]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:21.233 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.233+0000 7fdc21ffb700 1 -- 172.21.15.196:0/3990820013 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdc24062750 con 0x7fdc140a2890 2024-03-18T23:06:21.234 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 msgr2=0x7fdc140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 0x7fdc140a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc18006fd0 tx=0x7fdc18008040 comp rx=0 tx=0).stop 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 msgr2=0x7fdc2419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00be60 tx=0x7fdc1c00be90 comp rx=0 tx=0).stop 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 shutdown_connections 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdc140a2890 0x7fdc140a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc24103ed0 0x7fdc2419fc10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc24102330 0x7fdc24198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.236 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 --2- 172.21.15.196:0/3990820013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc24103530 0x7fdc24198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:21.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 >> 172.21.15.196:0/3990820013 conn(0x7fdc240fd8f0 msgr2=0x7fdc24106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:21.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 shutdown_connections 2024-03-18T23:06:21.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:21.235+0000 7fdc2ab5e700 1 -- 172.21.15.196:0/3990820013 wait complete. 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:21 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:21 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T23:06:21.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:21 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:21.914 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.11.service 2024-03-18T23:06:21.916 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-18T23:06:21.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T23:06:22.169 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:23.389 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:06:23.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: osdmap e81: 12 total, 11 up, 12 in 2024-03-18T23:06:23.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:23.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.390 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:23 smithi196 ceph-mon[67089]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:23.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:06:23.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: osdmap e81: 12 total, 11 up, 12 in 2024-03-18T23:06:23.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:23.478 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.479 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:23 smithi132 ceph-mon[32020]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:23.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T23:06:23.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: from='osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: osdmap e81: 12 total, 11 up, 12 in 2024-03-18T23:06:23.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:23.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-18T23:06:23.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:23 smithi144 ceph-mon[66781]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:23.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 -- 172.21.15.132:0/2273697233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44102370 msgr2=0x7efe44102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 --2- 172.21.15.132:0/2273697233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44102370 0x7efe44102790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe380099c0 tx=0x7efe38009cd0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 -- 172.21.15.132:0/2273697233 shutdown_connections 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 --2- 172.21.15.132:0/2273697233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103f10 0x7efe4410a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 --2- 172.21.15.132:0/2273697233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44102370 0x7efe44102790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 --2- 172.21.15.132:0/2273697233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44103570 0x7efe441039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 -- 172.21.15.132:0/2273697233 >> 172.21.15.132:0/2273697233 conn(0x7efe440fd910 msgr2=0x7efe440ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 -- 172.21.15.132:0/2273697233 shutdown_connections 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.789+0000 7efe4912c700 1 -- 172.21.15.132:0/2273697233 wait complete. 2024-03-18T23:06:23.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 Processor -- start 2024-03-18T23:06:23.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 -- start start 2024-03-18T23:06:23.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:23.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103570 0x7efe44198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:23.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44103f10 0x7efe4419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:23.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410df60 con 0x7efe44103f10 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410dde0 con 0x7efe44103570 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4912c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe4410e0e0 con 0x7efe44102370 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60624/0 (socket says 172.21.15.132:60624) 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 -- 172.21.15.132:0/1581505796 learned_addr learned my addr 172.21.15.132:0/1581505796 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4259c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103570 0x7efe44198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe4359e700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44103f10 0x7efe4419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 -- 172.21.15.132:0/1581505796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103570 msgr2=0x7efe44198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103570 0x7efe44198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 -- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44103f10 msgr2=0x7efe4419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44103f10 0x7efe4419fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.790+0000 7efe42d9d700 1 -- 172.21.15.132:0/1581505796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441a01b0 con 0x7efe44102370 2024-03-18T23:06:23.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe42d9d700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe3800fab0 tx=0x7efe3800fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38027070 con 0x7efe44102370 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe44074540 con 0x7efe44102370 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3800fd00 con 0x7efe44102370 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380173f0 con 0x7efe44102370 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.791+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe44074a70 con 0x7efe44102370 2024-03-18T23:06:23.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.792+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe44068fd0 con 0x7efe44102370 2024-03-18T23:06:23.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.793+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7efe3802a070 con 0x7efe44102370 2024-03-18T23:06:23.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.794+0000 7efe2bfff700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 0x7efe2c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:23.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.794+0000 7efe4259c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 0x7efe2c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:23.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.794+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7efe380cf8b0 con 0x7efe44102370 2024-03-18T23:06:23.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.794+0000 7efe4259c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 0x7efe2c0a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe34010d80 tx=0x7efe34012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:23.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.800+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe38031070 con 0x7efe44102370 2024-03-18T23:06:23.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.995+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efe4402dbb0 con 0x7efe44102370 2024-03-18T23:06:23.997 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.996+0000 7efe2bfff700 1 -- 172.21.15.132:0/1581505796 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7efe38013070 con 0x7efe44102370 2024-03-18T23:06:23.997 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:23.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 msgr2=0x7efe2c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:23.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 0x7efe2c0a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe34010d80 tx=0x7efe34012040 comp rx=0 tx=0).stop 2024-03-18T23:06:23.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 msgr2=0x7efe44198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe3800fab0 tx=0x7efe3800fae0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 shutdown_connections 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efe2c0a2890 0x7efe2c0a4d50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe44103f10 0x7efe4419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe44102370 0x7efe44198550 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 --2- 172.21.15.132:0/1581505796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe44103570 0x7efe44198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.998+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 >> 172.21.15.132:0/1581505796 conn(0x7efe440fd910 msgr2=0x7efe44106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.999+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 shutdown_connections 2024-03-18T23:06:24.000 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:23.999+0000 7efe4912c700 1 -- 172.21.15.132:0/1581505796 wait complete. 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: osdmap e82: 12 total, 11 up, 12 in 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1581505796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:24.289 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:24 smithi132 ceph-mon[32020]: from='osd.11 ' entity='osd.11' 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: osdmap e82: 12 total, 11 up, 12 in 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1581505796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:24.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-mon[67089]: from='osd.11 ' entity='osd.11' 2024-03-18T23:06:24.475 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:06:24.112+0000 7f8d2dd9c700 -1 osd.11 0 waiting for initial osdmap 2024-03-18T23:06:24.475 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:06:24 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:06:24.252+0000 7f8d25b87700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T23:06:24.586 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":82,"num_osds":12,"num_up_osds":11,"osd_up_since":1710803159,"num_in_osds":12,"osd_in_since":1710803163,"num_remapped_pgs":0} 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: osdmap e82: 12 total, 11 up, 12 in 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1581505796' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:24 smithi144 ceph-mon[66781]: from='osd.11 ' entity='osd.11' 2024-03-18T23:06:25.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: purged_snaps scrub starts 2024-03-18T23:06:25.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: purged_snaps scrub ok 2024-03-18T23:06:25.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: Detected new or changed devices on smithi196 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 4270M 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:25.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:25 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: purged_snaps scrub starts 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: purged_snaps scrub ok 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: Detected new or changed devices on smithi196 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 4270M 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:25.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:25 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.613 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: purged_snaps scrub starts 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: purged_snaps scrub ok 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: Detected new or changed devices on smithi196 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 4270M 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:25 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:25.827 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:26 smithi196 ceph-mon[67089]: osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] boot 2024-03-18T23:06:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:26 smithi196 ceph-mon[67089]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:26.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:26 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:26 smithi144 ceph-mon[66781]: osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] boot 2024-03-18T23:06:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:26 smithi144 ceph-mon[66781]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:26 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:26 smithi132 ceph-mon[32020]: osd.11 [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] boot 2024-03-18T23:06:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:26 smithi132 ceph-mon[32020]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T23:06:26.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:26 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:06:27.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:27 smithi132 ceph-mon[32020]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:27.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/78599115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4103fa0 msgr2=0x7f0db410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:27.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/78599115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4103fa0 0x7f0db410a850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00b0a0 tx=0x7f0dac00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/78599115 shutdown_connections 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/78599115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4100620 0x7f0db4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/78599115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4103fa0 0x7f0db410a850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/78599115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103600 0x7f0db4103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/78599115 >> 172.21.15.132:0/78599115 conn(0x7f0db40fbfa0 msgr2=0x7f0db40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/78599115 shutdown_connections 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/78599115 wait complete. 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.371+0000 7f0dbb1a7700 1 Processor -- start 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 -- start start 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4103600 0x7f0db4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:27.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103fa0 0x7f0db419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410e030 con 0x7f0db4100620 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410deb0 con 0x7f0db4103600 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0db410e1b0 con 0x7f0db4103fa0 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44478/0 (socket says 172.21.15.132:44478) 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 -- 172.21.15.132:0/220350775 learned_addr learned my addr 172.21.15.132:0/220350775 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db3fff700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4103600 0x7f0db4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db9744700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103fa0 0x7f0db419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 -- 172.21.15.132:0/220350775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103fa0 msgr2=0x7f0db419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103fa0 0x7f0db419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 -- 172.21.15.132:0/220350775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4103600 msgr2=0x7f0db4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4103600 0x7f0db4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 -- 172.21.15.132:0/220350775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db41a0210 con 0x7f0db4100620 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db8f43700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b530 tx=0x7f0da800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8015020 con 0x7f0db4100620 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8011d10 con 0x7f0db4100620 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.372+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db41a0500 con 0x7f0db4100620 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.373+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800f2e0 con 0x7f0db4100620 2024-03-18T23:06:27.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.373+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db41a0b10 con 0x7f0db4100620 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.374+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db4068fd0 con 0x7f0db4100620 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.375+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0da8018020 con 0x7f0db4100620 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.375+0000 7f0db1ffb700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 0x7f0da40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.375+0000 7f0db3fff700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 0x7f0da40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.375+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0da80cdd20 con 0x7f0db4100620 2024-03-18T23:06:27.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.376+0000 7f0db3fff700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 0x7f0da40a4dd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:27.380 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.380+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0da801f020 con 0x7f0db4100620 2024-03-18T23:06:27.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:27 smithi196 ceph-mon[67089]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:27.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.572+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0db402dc30 con 0x7f0db4100620 2024-03-18T23:06:27.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.572+0000 7f0db1ffb700 1 -- 172.21.15.132:0/220350775 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7f0da8010e90 con 0x7f0db4100620 2024-03-18T23:06:27.573 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 msgr2=0x7f0da40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 0x7f0da40a4dd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0008040 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 msgr2=0x7f0db41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b530 tx=0x7f0da800b8f0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 shutdown_connections 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0da40a2910 0x7f0da40a4dd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0db4103fa0 0x7f0db419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0db4100620 0x7f0db41985b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 --2- 172.21.15.132:0/220350775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db4103600 0x7f0db4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:27.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 >> 172.21.15.132:0/220350775 conn(0x7f0db40fbfa0 msgr2=0x7f0db4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:27.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 shutdown_connections 2024-03-18T23:06:27.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:27.575+0000 7f0dbb1a7700 1 -- 172.21.15.132:0/220350775 wait complete. 2024-03-18T23:06:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:27 smithi144 ceph-mon[66781]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:28.166 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1710803185,"num_in_osds":12,"osd_in_since":1710803163,"num_remapped_pgs":0} 2024-03-18T23:06:28.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:06:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:28.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:28 smithi196 ceph-mon[67089]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:28.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:28 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/220350775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:28.565 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:28 smithi132 ceph-mon[32020]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:28.565 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:28 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/220350775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:28.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:28 smithi144 ceph-mon[66781]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T23:06:28.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:28 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/220350775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T23:06:29.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:29 smithi196 ceph-mon[67089]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:29.550 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:29 smithi132 ceph-mon[32020]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:29.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:29 smithi144 ceph-mon[66781]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.059+0000 7fc814b18700 1 -- 172.21.15.132:0/3653132668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810108470 msgr2=0x7fc810108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.059+0000 7fc814b18700 1 --2- 172.21.15.132:0/3653132668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810108470 0x7fc810108850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc800009990 tx=0x7fc800009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.059+0000 7fc814b18700 1 -- 172.21.15.132:0/3653132668 shutdown_connections 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 --2- 172.21.15.132:0/3653132668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 0x7fc81010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 --2- 172.21.15.132:0/3653132668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810108470 0x7fc810108850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 --2- 172.21.15.132:0/3653132668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810102390 0x7fc810102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 -- 172.21.15.132:0/3653132668 >> 172.21.15.132:0/3653132668 conn(0x7fc8100793d0 msgr2=0x7fc8100797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 -- 172.21.15.132:0/3653132668 shutdown_connections 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 -- 172.21.15.132:0/3653132668 wait complete. 2024-03-18T23:06:30.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 Processor -- start 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.060+0000 7fc814b18700 1 -- start start 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 0x7fc810074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810108470 0x7fc810075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc810112650 con 0x7fc810102390 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101124d0 con 0x7fc810102d50 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc814b18700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101127d0 con 0x7fc810108470 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:44490/0 (socket says 172.21.15.132:44490) 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 -- 172.21.15.132:0/3371694393 learned_addr learned my addr 172.21.15.132:0/3371694393 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc805d9b700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 0x7fc810074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80ed9d700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810108470 0x7fc810075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:30.065 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 -- 172.21.15.132:0/3371694393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810108470 msgr2=0x7fc810075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810108470 0x7fc810075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 -- 172.21.15.132:0/3371694393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 msgr2=0x7fc810074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 0x7fc810074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 -- 172.21.15.132:0/3371694393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8101a7020 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc80e59c700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc800000c00 tx=0x7fc800003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.061+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800027070 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.062+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc800004110 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.062+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8101a72b0 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.062+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80000f460 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.062+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a77e0 con 0x7fc810102390 2024-03-18T23:06:30.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.062+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810068fd0 con 0x7fc810102390 2024-03-18T23:06:30.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.064+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc80002a070 con 0x7fc810102390 2024-03-18T23:06:30.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.064+0000 7fc807fff700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 0x7fc7fc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:30.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.064+0000 7fc805d9b700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 0x7fc7fc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:30.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.064+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc8000cf960 con 0x7fc810102390 2024-03-18T23:06:30.067 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.065+0000 7fc805d9b700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 0x7fc7fc0a4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8100799b0 tx=0x7fc7f8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:30.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.068+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc800031070 con 0x7fc810102390 2024-03-18T23:06:30.264 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.263+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc81002dbb0 con 0x7fc810102390 2024-03-18T23:06:30.265 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.264+0000 7fc807fff700 1 -- 172.21.15.132:0/3371694393 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18503 (secure 0 0 0) 0x7fc800013070 con 0x7fc810102390 2024-03-18T23:06:30.265 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:30.266 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":84,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:06:27.206168+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:30.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 msgr2=0x7fc7fc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:30.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 0x7fc7fc0a4dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8100799b0 tx=0x7fc7f8009190 comp rx=0 tx=0).stop 2024-03-18T23:06:30.267 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 msgr2=0x7fc810074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc800000c00 tx=0x7fc800003680 comp rx=0 tx=0).stop 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 shutdown_connections 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7fc0a2910 0x7fc7fc0a4dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc810108470 0x7fc810075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810102390 0x7fc810074610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 --2- 172.21.15.132:0/3371694393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc810102d50 0x7fc810074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 >> 172.21.15.132:0/3371694393 conn(0x7fc8100793d0 msgr2=0x7fc810101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 shutdown_connections 2024-03-18T23:06:30.268 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:30.267+0000 7fc814b18700 1 -- 172.21.15.132:0/3371694393 wait complete. 2024-03-18T23:06:30.598 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:30 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3371694393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:30.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:30 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3371694393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:30.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:30 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3371694393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:30.879 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:06:30.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-18T23:06:31.116 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:06:31.569 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:31 smithi132 ceph-mon[32020]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:31.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:31 smithi144 ceph-mon[66781]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:31 smithi196 ceph-mon[67089]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 -- 172.21.15.132:0/313945783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 msgr2=0x7f5ec010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 --2- 172.21.15.132:0/313945783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec010d840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800b0a0 tx=0x7f5eb800b3b0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 -- 172.21.15.132:0/313945783 shutdown_connections 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 --2- 172.21.15.132:0/313945783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec010d840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 --2- 172.21.15.132:0/313945783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0106a20 0x7f5ec0106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 --2- 172.21.15.132:0/313945783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0100c40 0x7f5ec01010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 -- 172.21.15.132:0/313945783 >> 172.21.15.132:0/313945783 conn(0x7f5ec0076540 msgr2=0x7f5ec0076950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:32.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 -- 172.21.15.132:0/313945783 shutdown_connections 2024-03-18T23:06:32.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.729+0000 7f5ec505a700 1 -- 172.21.15.132:0/313945783 wait complete. 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 Processor -- start 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 -- start start 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0100c40 0x7f5ec01a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 0x7f5ec019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0110460 con 0x7f5ec0101600 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01102e0 con 0x7f5ec0106a20 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ec505a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01105e0 con 0x7f5ec0100c40 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.730+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38360/0 (socket says 172.21.15.132:38360) 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 -- 172.21.15.132:0/1583684104 learned_addr learned my addr 172.21.15.132:0/1583684104 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:32.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebed9d700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0100c40 0x7f5ec01a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebf59e700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 0x7f5ec019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 -- 172.21.15.132:0/1583684104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0100c40 msgr2=0x7f5ec01a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0100c40 0x7f5ec01a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 -- 172.21.15.132:0/1583684104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 msgr2=0x7f5ec019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 0x7f5ec019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 -- 172.21.15.132:0/1583684104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec019b130 con 0x7f5ec0101600 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebf59e700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 0x7f5ec019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ebe59c700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000ddc0 tx=0x7f5eb000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb001b070 con 0x7f5ec0101600 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb000ad40 con 0x7f5ec0101600 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec019b420 con 0x7f5ec0101600 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0016860 con 0x7f5ec0101600 2024-03-18T23:06:32.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.731+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01a76e0 con 0x7f5ec0101600 2024-03-18T23:06:32.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.732+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec0068fd0 con 0x7f5ec0101600 2024-03-18T23:06:32.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.734+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5eb001e070 con 0x7f5ec0101600 2024-03-18T23:06:32.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.735+0000 7f5ea7fff700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 0x7f5ea80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:32.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.735+0000 7f5ebed9d700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 0x7f5ea80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:32.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.735+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5eb00ce1a0 con 0x7f5ec0101600 2024-03-18T23:06:32.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.735+0000 7f5ebed9d700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 0x7f5ea80a4d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ec019bb10 tx=0x7f5eb400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:32.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.738+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5eb009b6e0 con 0x7f5ec0101600 2024-03-18T23:06:32.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.934+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ec019bf00 con 0x7f5ec0101600 2024-03-18T23:06:32.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.935+0000 7f5ea7fff700 1 -- 172.21.15.132:0/1583684104 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f5eb009afe0 con 0x7f5ec0101600 2024-03-18T23:06:32.935 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 msgr2=0x7f5ea80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 0x7f5ea80a4d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ec019bb10 tx=0x7f5eb400b3c0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 msgr2=0x7f5ec01a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000ddc0 tx=0x7f5eb000a530 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 shutdown_connections 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5ea80a2890 0x7f5ea80a4d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ec0106a20 0x7f5ec019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ec0100c40 0x7f5ec01a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 --2- 172.21.15.132:0/1583684104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec0101600 0x7f5ec01a0d50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:32.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 >> 172.21.15.132:0/1583684104 conn(0x7f5ec0076540 msgr2=0x7f5ec00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:32.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 shutdown_connections 2024-03-18T23:06:32.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:32.937+0000 7f5ec505a700 1 -- 172.21.15.132:0/1583684104 wait complete. 2024-03-18T23:06:33.286 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:32 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1583684104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:33.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:32 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1583684104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:33.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:32 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1583684104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:33.600 INFO:tasks.cephadm:Adding mds.a on smithi132 2024-03-18T23:06:33.600 INFO:tasks.cephadm:Adding mds.d on smithi132 2024-03-18T23:06:33.600 INFO:tasks.cephadm:Adding mds.g on smithi132 2024-03-18T23:06:33.600 INFO:tasks.cephadm:Adding mds.j on smithi132 2024-03-18T23:06:33.600 INFO:tasks.cephadm:Adding mds.b on smithi144 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.e on smithi144 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.h on smithi144 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.k on smithi144 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-03-18T23:06:33.601 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-03-18T23:06:33.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-03-18T23:06:33.852 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:06:34.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:33 smithi196 ceph-mon[67089]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:33 smithi144 ceph-mon[66781]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:34.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:33 smithi132 ceph-mon[32020]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 -- 172.21.15.196:0/1141137173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18102330 msgr2=0x7f8f18102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 --2- 172.21.15.196:0/1141137173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18102330 0x7f8f18102750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009990 tx=0x7f8f08009ca0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 -- 172.21.15.196:0/1141137173 shutdown_connections 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 --2- 172.21.15.196:0/1141137173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103ed0 0x7f8f1810a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 --2- 172.21.15.196:0/1141137173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18102330 0x7f8f18102750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 --2- 172.21.15.196:0/1141137173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18103530 0x7f8f18103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.411+0000 7f8f2006a700 1 -- 172.21.15.196:0/1141137173 >> 172.21.15.196:0/1141137173 conn(0x7f8f180fd8f0 msgr2=0x7f8f180ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- 172.21.15.196:0/1141137173 shutdown_connections 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- 172.21.15.196:0/1141137173 wait complete. 2024-03-18T23:06:35.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 Processor -- start 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- start start 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18102330 0x7f8f180760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103530 0x7f8f18074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810df80 con 0x7f8f18102330 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810de00 con 0x7f8f18103530 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.412+0000 7f8f2006a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810e100 con 0x7f8f18103ed0 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:35.416 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:34958/0 (socket says 172.21.15.196:34958) 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 -- 172.21.15.196:0/3784260748 learned_addr learned my addr 172.21.15.196:0/3784260748 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1de06700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18102330 0x7f8f180760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1d605700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103530 0x7f8f18074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 -- 172.21.15.196:0/3784260748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103530 msgr2=0x7f8f18074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103530 0x7f8f18074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 -- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18102330 msgr2=0x7f8f180760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18102330 0x7f8f180760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 -- 172.21.15.196:0/3784260748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180754f0 con 0x7f8f18103ed0 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f1e607700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b970 tx=0x7f8f1000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:35.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10019070 con 0x7f8f18103ed0 2024-03-18T23:06:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1000ce10 con 0x7f8f18103ed0 2024-03-18T23:06:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181a2750 con 0x7f8f18103ed0 2024-03-18T23:06:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10007370 con 0x7f8f18103ed0 2024-03-18T23:06:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.413+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181a2c90 con 0x7f8f18103ed0 2024-03-18T23:06:35.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.414+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f18068fd0 con 0x7f8f18103ed0 2024-03-18T23:06:35.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.415+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f8f1001c070 con 0x7f8f18103ed0 2024-03-18T23:06:35.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.416+0000 7f8f0effd700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 0x7f8f040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:35.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.416+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f8f100cd920 con 0x7f8f18103ed0 2024-03-18T23:06:35.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.416+0000 7f8f1de06700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 0x7f8f040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:35.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.416+0000 7f8f1de06700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 0x7f8f040a4d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003890 tx=0x7f8f08003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:35.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.420+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8f1009ae60 con 0x7f8f18103ed0 2024-03-18T23:06:35.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.626+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f8f18062850 con 0x7f8f040a2890 2024-03-18T23:06:35.636 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.636+0000 7f8f0effd700 1 -- 172.21.15.196:0/3784260748 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8f18062850 con 0x7f8f040a2890 2024-03-18T23:06:35.636 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-03-18T23:06:35.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 msgr2=0x7f8f040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:35.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 0x7f8f040a4d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003890 tx=0x7f8f08003960 comp rx=0 tx=0).stop 2024-03-18T23:06:35.638 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 msgr2=0x7f8f18074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b970 tx=0x7f8f1000bc80 comp rx=0 tx=0).stop 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 shutdown_connections 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8f040a2890 0x7f8f040a4d50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f18103ed0 0x7f8f18074c40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f18102330 0x7f8f180760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 --2- 172.21.15.196:0/3784260748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f18103530 0x7f8f18074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 >> 172.21.15.196:0/3784260748 conn(0x7f8f180fd8f0 msgr2=0x7f8f18106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 shutdown_connections 2024-03-18T23:06:35.639 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-18T23:06:35.638+0000 7f8f2006a700 1 -- 172.21.15.196:0/3784260748 wait complete. 2024-03-18T23:06:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:36 smithi196 ceph-mon[67089]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:36.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:36 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:36.297 DEBUG:teuthology.orchestra.run.smithi132:mds.a> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.a.service 2024-03-18T23:06:36.300 DEBUG:teuthology.orchestra.run.smithi132:mds.d> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.d.service 2024-03-18T23:06:36.343 DEBUG:teuthology.orchestra.run.smithi132:mds.g> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.g.service 2024-03-18T23:06:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:36 smithi144 ceph-mon[66781]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:36 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:36 smithi132 ceph-mon[32020]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:36 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:06:36.366 DEBUG:teuthology.orchestra.run.smithi132:mds.j> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.j.service 2024-03-18T23:06:36.409 DEBUG:teuthology.orchestra.run.smithi144:mds.b> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.b.service 2024-03-18T23:06:36.412 DEBUG:teuthology.orchestra.run.smithi144:mds.e> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.e.service 2024-03-18T23:06:36.455 DEBUG:teuthology.orchestra.run.smithi144:mds.h> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.h.service 2024-03-18T23:06:36.498 DEBUG:teuthology.orchestra.run.smithi144:mds.k> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.k.service 2024-03-18T23:06:36.541 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.c.service 2024-03-18T23:06:36.543 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.f.service 2024-03-18T23:06:36.587 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.i.service 2024-03-18T23:06:36.630 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.l.service 2024-03-18T23:06:36.673 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-18T23:06:36.674 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:06:36.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 -- 172.21.15.132:0/2937500876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 msgr2=0x7f5d681046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:36.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/2937500876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d681046e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009900 tx=0x7f5d5802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:36.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 -- 172.21.15.132:0/2937500876 shutdown_connections 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/2937500876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68107550 0x7f5d68109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/2937500876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68104c20 0x7f5d68107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/2937500876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d681046e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.785+0000 7f5d6dd69640 1 -- 172.21.15.132:0/2937500876 >> 172.21.15.132:0/2937500876 conn(0x7f5d680fbd00 msgr2=0x7f5d680fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- 172.21.15.132:0/2937500876 shutdown_connections 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- 172.21.15.132:0/2937500876 wait complete. 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 Processor -- start 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- start start 2024-03-18T23:06:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68104c20 0x7f5d68198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68107550 0x7f5d6819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d4f0 con 0x7f5d681022f0 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d370 con 0x7f5d68107550 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.786+0000 7f5d6dd69640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810d670 con 0x7f5d68104c20 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38376/0 (socket says 172.21.15.132:38376) 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 -- 172.21.15.132:0/3797797338 learned_addr learned my addr 172.21.15.132:0/3797797338 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d66ffd640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68104c20 0x7f5d68198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 -- 172.21.15.132:0/3797797338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68104c20 msgr2=0x7f5d68198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68104c20 0x7f5d68198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 -- 172.21.15.132:0/3797797338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68107550 msgr2=0x7f5d6819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68107550 0x7f5d6819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 -- 172.21.15.132:0/3797797338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6819f700 con 0x7f5d681022f0 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d677fe640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009230 tx=0x7f5d58005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58041070 con 0x7f5d681022f0 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d580054e0 con 0x7f5d681022f0 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6819f990 con 0x7f5d681022f0 2024-03-18T23:06:36.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58034440 con 0x7f5d681022f0 2024-03-18T23:06:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.787+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6819fe80 con 0x7f5d681022f0 2024-03-18T23:06:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.788+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d680fd7f0 con 0x7f5d681022f0 2024-03-18T23:06:36.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.789+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5d58044070 con 0x7f5d681022f0 2024-03-18T23:06:36.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.790+0000 7f5d64ff9640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 0x7f5d500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:36.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.790+0000 7f5d66ffd640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 0x7f5d500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:36.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.790+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5d580f0600 con 0x7f5d681022f0 2024-03-18T23:06:36.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.790+0000 7f5d66ffd640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 0x7f5d500a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d68198ff0 tx=0x7f5d5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:36.795 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.794+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5d580bdb40 con 0x7f5d681022f0 2024-03-18T23:06:36.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.970+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d6802d450 con 0x7f5d681022f0 2024-03-18T23:06:36.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.972+0000 7f5d64ff9640 1 -- 172.21.15.132:0/3797797338 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18503 (secure 0 0 0) 0x7f5d580bd440 con 0x7f5d681022f0 2024-03-18T23:06:36.973 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:36.973 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":84,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:06:27.206168+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:06:36.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 msgr2=0x7f5d500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:36.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 0x7f5d500a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5d68198ff0 tx=0x7f5d5c00a2d0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 msgr2=0x7f5d68197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009230 tx=0x7f5d58005dd0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 shutdown_connections 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5d500a2800 0x7f5d500a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d68107550 0x7f5d6819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d68104c20 0x7f5d68198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 --2- 172.21.15.132:0/3797797338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d681022f0 0x7f5d68197ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:36.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 >> 172.21.15.132:0/3797797338 conn(0x7f5d680fbd00 msgr2=0x7f5d68102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:36.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 shutdown_connections 2024-03-18T23:06:36.977 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:36.974+0000 7f5d6dd69640 1 -- 172.21.15.132:0/3797797338 wait complete. 2024-03-18T23:06:36.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:06:36.987 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:06:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 -- 172.21.15.132:0/2679227253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c1033e0 msgr2=0x7f7b6c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 --2- 172.21.15.132:0/2679227253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c109c70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009900 tx=0x7f7b6802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 -- 172.21.15.132:0/2679227253 shutdown_connections 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 --2- 172.21.15.132:0/2679227253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c109c70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 --2- 172.21.15.132:0/2679227253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c102a20 0x7f7b6c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 --2- 172.21.15.132:0/2679227253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c101830 0x7f7b6c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 -- 172.21.15.132:0/2679227253 >> 172.21.15.132:0/2679227253 conn(0x7f7b6c0fcdc0 msgr2=0x7f7b6c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 -- 172.21.15.132:0/2679227253 shutdown_connections 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.091+0000 7f7b7337e640 1 -- 172.21.15.132:0/2679227253 wait complete. 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 Processor -- start 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 -- start start 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c102a20 0x7f7b6c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d4b0 con 0x7f7b6c101830 2024-03-18T23:06:37.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d330 con 0x7f7b6c1033e0 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7337e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c10d630 con 0x7f7b6c102a20 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38390/0 (socket says 172.21.15.132:38390) 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 -- 172.21.15.132:0/3407002631 learned_addr learned my addr 172.21.15.132:0/3407002631 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7191b640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b70919640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c102a20 0x7f7b6c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 -- 172.21.15.132:0/3407002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c102a20 msgr2=0x7f7b6c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c102a20 0x7f7b6c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 -- 172.21.15.132:0/3407002631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c1033e0 msgr2=0x7f7b6c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 -- 172.21.15.132:0/3407002631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c19f710 con 0x7f7b6c101830 2024-03-18T23:06:37.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.092+0000 7f7b7111a640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400eb70 tx=0x7f7b6400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.095+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64003b70 con 0x7f7b6c101830 2024-03-18T23:06:37.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.096+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6400ee70 con 0x7f7b6c101830 2024-03-18T23:06:37.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.096+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c19fa00 con 0x7f7b6c101830 2024-03-18T23:06:37.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.096+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64010830 con 0x7f7b6c101830 2024-03-18T23:06:37.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.096+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c19ff00 con 0x7f7b6c101830 2024-03-18T23:06:37.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.097+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c108b50 con 0x7f7b6c101830 2024-03-18T23:06:37.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.097+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f7b64004020 con 0x7f7b6c101830 2024-03-18T23:06:37.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.098+0000 7f7b627fc640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 0x7f7b580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.098+0000 7f7b70919640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 0x7f7b580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.098+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7b640cec30 con 0x7f7b6c101830 2024-03-18T23:06:37.099 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.098+0000 7f7b70919640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 0x7f7b580a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c199000 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.102+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7b6409c170 con 0x7f7b6c101830 2024-03-18T23:06:37.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.278+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b6c02d470 con 0x7f7b6c101830 2024-03-18T23:06:37.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.278+0000 7f7b627fc640 1 -- 172.21.15.132:0/3407002631 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f7b6409ba70 con 0x7f7b6c101830 2024-03-18T23:06:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 msgr2=0x7f7b580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 0x7f7b580a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c199000 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 msgr2=0x7f7b6c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400eb70 tx=0x7f7b6400c6a0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 shutdown_connections 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7b580a2800 0x7f7b580a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b6c1033e0 0x7f7b6c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b6c102a20 0x7f7b6c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 --2- 172.21.15.132:0/3407002631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b6c101830 0x7f7b6c197af0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 >> 172.21.15.132:0/3407002631 conn(0x7f7b6c0fcdc0 msgr2=0x7f7b6c105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 shutdown_connections 2024-03-18T23:06:37.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.280+0000 7f7b7337e640 1 -- 172.21.15.132:0/3407002631 wait complete. 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.291 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3797797338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:37.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:37.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:37.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-03-18T23:06:37.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:37.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:37.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:37.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:37.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.361 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3797797338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- 172.21.15.132:0/46218182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994107580 msgr2=0x7fb994109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 --2- 172.21.15.132:0/46218182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994107580 0x7fb994109970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009900 tx=0x7fb98c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- 172.21.15.132:0/46218182 shutdown_connections 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 --2- 172.21.15.132:0/46218182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994107580 0x7fb994109970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 --2- 172.21.15.132:0/46218182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb994107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 --2- 172.21.15.132:0/46218182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994102320 0x7fb994104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- 172.21.15.132:0/46218182 >> 172.21.15.132:0/46218182 conn(0x7fb9940fbcb0 msgr2=0x7fb9940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- 172.21.15.132:0/46218182 shutdown_connections 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- 172.21.15.132:0/46218182 wait complete. 2024-03-18T23:06:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 Processor -- start 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.394+0000 7fb99b644640 1 -- start start 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb9941980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994107580 0x7fb99419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d4e0 con 0x7fb994102320 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d360 con 0x7fb994104c50 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d660 con 0x7fb994107580 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38416/0 (socket says 172.21.15.132:38416) 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 -- 172.21.15.132:0/1123890027 learned_addr learned my addr 172.21.15.132:0/1123890027 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:37.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb999be1640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994107580 0x7fb99419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb998bdf640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb9941980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 -- 172.21.15.132:0/1123890027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994107580 msgr2=0x7fb99419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994107580 0x7fb99419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 -- 172.21.15.132:0/1123890027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 msgr2=0x7fb9941980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb9941980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 -- 172.21.15.132:0/1123890027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99419f730 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb998bdf640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb9941980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb9993e0640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb99000eb90 tx=0x7fb99000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990004d20 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99419fa20 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99000ee90 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990010830 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.395+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99419ff70 con 0x7fb994102320 2024-03-18T23:06:37.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.396+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9940fd7a0 con 0x7fb994102320 2024-03-18T23:06:37.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.397+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb990005020 con 0x7fb994102320 2024-03-18T23:06:37.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.398+0000 7fb98a7fc640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 0x7fb9800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:37.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.398+0000 7fb998bdf640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 0x7fb9800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:37.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.398+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb9900ceca0 con 0x7fb994102320 2024-03-18T23:06:37.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.398+0000 7fb998bdf640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 0x7fb9800a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb984005d00 tx=0x7fb984009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:37.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.402+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb99009c1e0 con 0x7fb994102320 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: Saving service mds.all spec with placement smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:37.435 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3797797338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:06:37.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.615+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb994062420 con 0x7fb994102320 2024-03-18T23:06:37.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.615+0000 7fb98a7fc640 1 -- 172.21.15.132:0/1123890027 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb99009bae0 con 0x7fb994102320 2024-03-18T23:06:37.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:37.617 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-18T23:06:37.617 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 1 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 msgr2=0x7fb9800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 0x7fb9800a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb984005d00 tx=0x7fb984009550 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 msgr2=0x7fb994197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb99000eb90 tx=0x7fb99000c6a0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 shutdown_connections 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb9800a2800 0x7fb9800a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb994107580 0x7fb99419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb994104c50 0x7fb9941980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 --2- 172.21.15.132:0/1123890027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb994102320 0x7fb994197b60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 >> 172.21.15.132:0/1123890027 conn(0x7fb9940fbcb0 msgr2=0x7fb994102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:37.619 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 shutdown_connections 2024-03-18T23:06:37.620 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:37.618+0000 7fb99b644640 1 -- 172.21.15.132:0/1123890027 wait complete. 2024-03-18T23:06:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:38 smithi144 ceph-mon[66781]: Deploying daemon mds.l on smithi196 2024-03-18T23:06:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:38 smithi144 ceph-mon[66781]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3407002631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:38.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1123890027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:38 smithi132 ceph-mon[32020]: Deploying daemon mds.l on smithi196 2024-03-18T23:06:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:38 smithi132 ceph-mon[32020]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3407002631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1123890027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:38.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:38 smithi196 ceph-mon[67089]: Deploying daemon mds.l on smithi196 2024-03-18T23:06:38.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:38 smithi196 ceph-mon[67089]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:38.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3407002631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:06:38.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1123890027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:38.724 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:38 smithi196 systemd[1]: Starting Ceph mds.l for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:39.110 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:38 smithi196 podman[97065]: 2024-03-18 23:06:38.662480969 +0000 UTC m=+0.017711342 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:39.110 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:38 smithi196 podman[97065]: 2024-03-18 23:06:38.765163136 +0000 UTC m=+0.120393551 container create 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T23:06:39.110 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 podman[97065]: 2024-03-18 23:06:39.109491485 +0000 UTC m=+0.464721864 container init 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-18T23:06:39.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 podman[97065]: 2024-03-18 23:06:39.112657 +0000 UTC m=+0.467887368 container start 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-18T23:06:39.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 ceph-mds[97084]: -- 172.21.15.196:0/3746266102 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e28acffa40 con 0x55e28acd3000 2024-03-18T23:06:39.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l[97080]: starting mds.l at 2024-03-18T23:06:39.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 bash[97065]: 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 2024-03-18T23:06:39.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:06:39 smithi196 systemd[1]: Started Ceph mds.l for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:39.630 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 -- 172.21.15.132:0/757661763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 msgr2=0x7f0c04104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/757661763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04104710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 -- 172.21.15.132:0/757661763 shutdown_connections 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/757661763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04107580 0x7f0c04109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/757661763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04104c50 0x7f0c04107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/757661763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04104710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 -- 172.21.15.132:0/757661763 >> 172.21.15.132:0/757661763 conn(0x7f0c040fbcd0 msgr2=0x7f0c040fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 -- 172.21.15.132:0/757661763 shutdown_connections 2024-03-18T23:06:39.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.742+0000 7f0c09a7e640 1 -- 172.21.15.132:0/757661763 wait complete. 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 Processor -- start 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 -- start start 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04104c50 0x7f0c041980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 0x7f0c0419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410d4d0 con 0x7f0c04102320 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410d350 con 0x7f0c04104c50 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c09a7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410d650 con 0x7f0c04107580 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:39.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38432/0 (socket says 172.21.15.132:38432) 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 -- 172.21.15.132:0/2078758580 learned_addr learned my addr 172.21.15.132:0/2078758580 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c02ffd640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04104c50 0x7f0c041980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c03fff640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 0x7f0c0419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 -- 172.21.15.132:0/2078758580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 msgr2=0x7f0c0419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 0x7f0c0419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 -- 172.21.15.132:0/2078758580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04104c50 msgr2=0x7f0c041980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04104c50 0x7f0c041980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c037fe640 1 -- 172.21.15.132:0/2078758580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0419f7c0 con 0x7f0c04102320 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.743+0000 7f0c03fff640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 0x7f0c0419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c037fe640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009230 tx=0x7f0bf402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4041070 con 0x7f0c04102320 2024-03-18T23:06:39.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0419fa50 con 0x7f0c04102320 2024-03-18T23:06:39.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4004450 con 0x7f0c04102320 2024-03-18T23:06:39.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4003e20 con 0x7f0c04102320 2024-03-18T23:06:39.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.744+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0419ff40 con 0x7f0c04102320 2024-03-18T23:06:39.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.745+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c040fd7c0 con 0x7f0c04102320 2024-03-18T23:06:39.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.746+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0bf4044070 con 0x7f0c04102320 2024-03-18T23:06:39.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.747+0000 7f0c00ff9640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 0x7f0bec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:39.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.747+0000 7f0c02ffd640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 0x7f0bec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:39.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.747+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0bf40f0200 con 0x7f0c04102320 2024-03-18T23:06:39.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.747+0000 7f0c02ffd640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 0x7f0bec0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c041990b0 tx=0x7f0bf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:39.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.752+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0bf40bd740 con 0x7f0c04102320 2024-03-18T23:06:39.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.963+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c0402d1f0 con 0x7f0c04102320 2024-03-18T23:06:39.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.964+0000 7f0c00ff9640 1 -- 172.21.15.132:0/2078758580 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f0bf40bd040 con 0x7f0c04102320 2024-03-18T23:06:39.964 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:39.965 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T23:06:39.965 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 2 2024-03-18T23:06:39.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 msgr2=0x7f0bec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 0x7f0bec0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c041990b0 tx=0x7f0bf8005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 msgr2=0x7f0c04197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009230 tx=0x7f0bf402c650 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 shutdown_connections 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0bec0a2800 0x7f0bec0a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c04107580 0x7f0c0419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c04104c50 0x7f0c041980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 --2- 172.21.15.132:0/2078758580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c04102320 0x7f0c04197ba0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 >> 172.21.15.132:0/2078758580 conn(0x7f0c040fbcd0 msgr2=0x7f0c04102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 shutdown_connections 2024-03-18T23:06:39.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:39.966+0000 7f0c09a7e640 1 -- 172.21.15.132:0/2078758580 wait complete. 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6832/3738348032,v1:172.21.15.196:6833/3738348032] up:boot 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: fsmap 1 up:standby 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:40.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2078758580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6832/3738348032,v1:172.21.15.196:6833/3738348032] up:boot 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: fsmap 1 up:standby 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:40.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:40.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:40.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2078758580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6832/3738348032,v1:172.21.15.196:6833/3738348032] up:boot 2024-03-18T23:06:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: fsmap 1 up:standby 2024-03-18T23:06:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:06:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:40.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:40.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:40.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:40.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2078758580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:41.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:41 smithi196 ceph-mon[67089]: Deploying daemon mds.k on smithi144 2024-03-18T23:06:41.506 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:41 smithi144 ceph-mon[66781]: Deploying daemon mds.k on smithi144 2024-03-18T23:06:41.507 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 systemd[1]: Starting Ceph mds.k for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:41.561 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:41 smithi132 ceph-mon[32020]: Deploying daemon mds.k on smithi144 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 podman[97160]: 2024-03-18 23:06:41.409196414 +0000 UTC m=+0.018101377 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 podman[97160]: 2024-03-18 23:06:41.52044957 +0000 UTC m=+0.129354531 container create 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 podman[97160]: 2024-03-18 23:06:41.714785444 +0000 UTC m=+0.323690400 container init 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 podman[97160]: 2024-03-18 23:06:41.717248471 +0000 UTC m=+0.326153420 container start 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 ceph-mds[97179]: -- 172.21.15.144:0/1540395027 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639282dfa40 con 0x5639282b2c00 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k[97175]: starting mds.k at 2024-03-18T23:06:41.790 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 bash[97160]: 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe 2024-03-18T23:06:41.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- 172.21.15.132:0/503095509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 msgr2=0x7f5720108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 --2- 172.21.15.132:0/503095509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f5720108b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5718009ca0 tx=0x7f571802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- 172.21.15.132:0/503095509 shutdown_connections 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 --2- 172.21.15.132:0/503095509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f5720108b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 --2- 172.21.15.132:0/503095509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720101930 0x7f5720101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 --2- 172.21.15.132:0/503095509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720100740 0x7f5720100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- 172.21.15.132:0/503095509 >> 172.21.15.132:0/503095509 conn(0x7f57200fbcb0 msgr2=0x7f57200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- 172.21.15.132:0/503095509 shutdown_connections 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- 172.21.15.132:0/503095509 wait complete. 2024-03-18T23:06:42.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 Processor -- start 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.085+0000 7f572647e640 1 -- start start 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 0x7f57200694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 0x7f572019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f572010c3a0 con 0x7f57201022f0 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f572010c220 con 0x7f5720100740 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f572010c520 con 0x7f5720101930 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57016/0 (socket says 172.21.15.132:57016) 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 -- 172.21.15.132:0/1276345594 learned_addr learned my addr 172.21.15.132:0/1276345594 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571ffff640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 0x7f57200694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571f7fe640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 0x7f572019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 -- 172.21.15.132:0/1276345594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 msgr2=0x7f572019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 0x7f572019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 -- 172.21.15.132:0/1276345594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 msgr2=0x7f57200694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 0x7f57200694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 -- 172.21.15.132:0/1276345594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57201a18c0 con 0x7f57201022f0 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571f7fe640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 0x7f572019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571ffff640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 0x7f57200694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f5724a1b640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f571802c700 tx=0x7f5718032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718039040 con 0x7f57201022f0 2024-03-18T23:06:42.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57201a1af0 con 0x7f57201022f0 2024-03-18T23:06:42.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5718044070 con 0x7f57201022f0 2024-03-18T23:06:42.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57180042a0 con 0x7f57201022f0 2024-03-18T23:06:42.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.086+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57201a2030 con 0x7f57201022f0 2024-03-18T23:06:42.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.087+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5720107990 con 0x7f57201022f0 2024-03-18T23:06:42.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.089+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5718003970 con 0x7f57201022f0 2024-03-18T23:06:42.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.089+0000 7f571d7fa640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 0x7f57080a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:42.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.089+0000 7f571ffff640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 0x7f57080a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:42.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.089+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f57180f1100 con 0x7f57201022f0 2024-03-18T23:06:42.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.090+0000 7f571ffff640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 0x7f57080a4cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5720101790 tx=0x7f571000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:42.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.097+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57180be7f0 con 0x7f57201022f0 2024-03-18T23:06:42.109 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:06:41 smithi144 systemd[1]: Started Ceph mds.k for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:42.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.318+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f572002d1f0 con 0x7f57201022f0 2024-03-18T23:06:42.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.319+0000 7f571d7fa640 1 -- 172.21.15.132:0/1276345594 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f57180be0f0 con 0x7f57201022f0 2024-03-18T23:06:42.319 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:42.319 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-18T23:06:42.319 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 3 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 msgr2=0x7f57080a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 0x7f57080a4cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5720101790 tx=0x7f571000a5e0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 msgr2=0x7f57201a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f571802c700 tx=0x7f5718032f20 comp rx=0 tx=0).stop 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 shutdown_connections 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f57080a2830 0x7f57080a4cf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57201022f0 0x7f57201a10d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5720101930 0x7f572019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 --2- 172.21.15.132:0/1276345594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5720100740 0x7f57200694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:42.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 >> 172.21.15.132:0/1276345594 conn(0x7f57200fbcb0 msgr2=0x7f5720104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:42.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 shutdown_connections 2024-03-18T23:06:42.322 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:42.321+0000 7f572647e640 1 -- 172.21.15.132:0/1276345594 wait complete. 2024-03-18T23:06:42.333 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:42.334 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:42 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:42 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:42.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:42.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:42.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:42 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:43.463 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:43 smithi132 ceph-mon[32020]: Deploying daemon mds.j on smithi132 2024-03-18T23:06:43.463 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:43 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6832/2374016108,v1:172.21.15.144:6833/2374016108] up:boot 2024-03-18T23:06:43.463 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:43 smithi132 ceph-mon[32020]: fsmap 2 up:standby 2024-03-18T23:06:43.463 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:43 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:06:43.463 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:43 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1276345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:43.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:43 smithi196 ceph-mon[67089]: Deploying daemon mds.j on smithi132 2024-03-18T23:06:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:43 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6832/2374016108,v1:172.21.15.144:6833/2374016108] up:boot 2024-03-18T23:06:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:43 smithi196 ceph-mon[67089]: fsmap 2 up:standby 2024-03-18T23:06:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:43 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:06:43.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:43 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1276345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:43 smithi144 ceph-mon[66781]: Deploying daemon mds.j on smithi132 2024-03-18T23:06:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:43 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6832/2374016108,v1:172.21.15.144:6833/2374016108] up:boot 2024-03-18T23:06:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:43 smithi144 ceph-mon[66781]: fsmap 2 up:standby 2024-03-18T23:06:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:43 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:06:43.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:43 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1276345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:44.115 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:43 smithi132 systemd[1]: Starting Ceph mds.j for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:44.336 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:44.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:44 smithi144 ceph-mon[66781]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:44.401 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:44 smithi132 ceph-mon[32020]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:44.401 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 podman[72261]: 2024-03-18 23:06:44.041289307 +0000 UTC m=+0.018627635 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:44.401 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 podman[72261]: 2024-03-18 23:06:44.157329668 +0000 UTC m=+0.134667988 container create c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:06:44.402 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 podman[72261]: 2024-03-18 23:06:44.401593265 +0000 UTC m=+0.378931585 container init c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 -- 172.21.15.132:0/1993566807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc100740 msgr2=0x7f58fc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 --2- 172.21.15.132:0/1993566807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc100740 0x7f58fc100b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009900 tx=0x7f58ec02bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 -- 172.21.15.132:0/1993566807 shutdown_connections 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 --2- 172.21.15.132:0/1993566807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 --2- 172.21.15.132:0/1993566807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc101930 0x7f58fc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 --2- 172.21.15.132:0/1993566807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc100740 0x7f58fc100b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 -- 172.21.15.132:0/1993566807 >> 172.21.15.132:0/1993566807 conn(0x7f58fc0fbcb0 msgr2=0x7f58fc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 -- 172.21.15.132:0/1993566807 shutdown_connections 2024-03-18T23:06:44.445 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.444+0000 7f5902027640 1 -- 172.21.15.132:0/1993566807 wait complete. 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 Processor -- start 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 -- start start 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 0x7f58fc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10c3b0 con 0x7f58fc101930 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10c230 con 0x7f58fc1022f0 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f5902027640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10c530 con 0x7f58fc100740 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57054/0 (socket says 172.21.15.132:57054) 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 -- 172.21.15.132:0/4218743236 learned_addr learned my addr 172.21.15.132:0/4218743236 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:44.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58fbfff640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58fb7fe640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 0x7f58fc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 -- 172.21.15.132:0/4218743236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 msgr2=0x7f58fc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 0x7f58fc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 -- 172.21.15.132:0/4218743236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 msgr2=0x7f58fc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58faffd640 1 -- 172.21.15.132:0/4218743236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc19f7d0 con 0x7f58fc101930 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58fb7fe640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 0x7f58fc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.445+0000 7f58fbfff640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f58faffd640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c970 tx=0x7f58f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007bb0 con 0x7f58fc101930 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc19fac0 con 0x7f58fc101930 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f0007d50 con 0x7f58fc101930 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000f450 con 0x7f58fc101930 2024-03-18T23:06:44.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.446+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc19ffc0 con 0x7f58fc101930 2024-03-18T23:06:44.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.447+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc1a0250 con 0x7f58fc101930 2024-03-18T23:06:44.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.448+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f58f0005020 con 0x7f58fc101930 2024-03-18T23:06:44.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.448+0000 7f58f8ff9640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 0x7f58e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:44.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.448+0000 7f58fb7fe640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 0x7f58e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:44.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.449+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f58f00ce380 con 0x7f58fc101930 2024-03-18T23:06:44.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.449+0000 7f58fb7fe640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 0x7f58e40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02c7b0 tx=0x7f58ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:44.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.453+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f58f0014030 con 0x7f58fc101930 2024-03-18T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:44 smithi196 ceph-mon[67089]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:44.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.676+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58fc1a0420 con 0x7f58fc101930 2024-03-18T23:06:44.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.677+0000 7f58f8ff9640 1 -- 172.21.15.132:0/4218743236 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f58f009b8c0 con 0x7f58fc101930 2024-03-18T23:06:44.678 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:44.678 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-18T23:06:44.678 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 3 2024-03-18T23:06:44.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 msgr2=0x7f58e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 0x7f58e40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58ec02c7b0 tx=0x7f58ec005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 msgr2=0x7f58fc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:44.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c970 tx=0x7f58f000ce30 comp rx=0 tx=0).stop 2024-03-18T23:06:44.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 shutdown_connections 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f58e40a2800 0x7f58e40a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58fc1022f0 0x7f58fc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58fc101930 0x7f58fc1980f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 --2- 172.21.15.132:0/4218743236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58fc100740 0x7f58fc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.679+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 >> 172.21.15.132:0/4218743236 conn(0x7f58fc0fbcb0 msgr2=0x7f58fc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.680+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 shutdown_connections 2024-03-18T23:06:44.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:44.680+0000 7f5902027640 1 -- 172.21.15.132:0/4218743236 wait complete. 2024-03-18T23:06:44.690 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 podman[72261]: 2024-03-18 23:06:44.405117553 +0000 UTC m=+0.382455882 container start c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:06:44.690 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 ceph-mds[72300]: -- 172.21.15.132:0/1515579031 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654f5015a40 con 0x5654f4fe9000 2024-03-18T23:06:44.690 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j[72276]: starting mds.j at 2024-03-18T23:06:44.690 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 bash[72261]: c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc 2024-03-18T23:06:44.690 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:06:44 smithi132 systemd[1]: Started Ceph mds.j for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:45.857 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: Deploying daemon mds.d on smithi132 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4218743236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.132:6834/1067641512,v1:172.21.15.132:6835/1067641512] up:boot 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: fsmap 3 up:standby 2024-03-18T23:06:45.858 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:45 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: Deploying daemon mds.d on smithi132 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4218743236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.132:6834/1067641512,v1:172.21.15.132:6835/1067641512] up:boot 2024-03-18T23:06:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: fsmap 3 up:standby 2024-03-18T23:06:45.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:45 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: Deploying daemon mds.d on smithi132 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4218743236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:45.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.132:6834/1067641512,v1:172.21.15.132:6835/1067641512] up:boot 2024-03-18T23:06:45.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: fsmap 3 up:standby 2024-03-18T23:06:45.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:45 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:06:46.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:46.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.815+0000 7f2cd311b640 1 -- 172.21.15.132:0/4259791491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc101930 msgr2=0x7f2ccc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.815+0000 7f2cd311b640 1 --2- 172.21.15.132:0/4259791491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc101930 0x7f2ccc101db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009900 tx=0x7f2cb802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.815+0000 7f2cd311b640 1 -- 172.21.15.132:0/4259791491 shutdown_connections 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.815+0000 7f2cd311b640 1 --2- 172.21.15.132:0/4259791491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 0x7f2ccc108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.815+0000 7f2cd311b640 1 --2- 172.21.15.132:0/4259791491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc101930 0x7f2ccc101db0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 --2- 172.21.15.132:0/4259791491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc100740 0x7f2ccc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- 172.21.15.132:0/4259791491 >> 172.21.15.132:0/4259791491 conn(0x7f2ccc0fbcb0 msgr2=0x7f2ccc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- 172.21.15.132:0/4259791491 shutdown_connections 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- 172.21.15.132:0/4259791491 wait complete. 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 Processor -- start 2024-03-18T23:06:46.817 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- start start 2024-03-18T23:06:46.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:46.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc101930 0x7f2ccc196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:46.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 0x7f2ccc19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:46.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10c3a0 con 0x7f2ccc100740 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10c220 con 0x7f2ccc101930 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.816+0000 7f2cd311b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10c520 con 0x7f2ccc1022f0 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57078/0 (socket says 172.21.15.132:57078) 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 -- 172.21.15.132:0/2186771212 learned_addr learned my addr 172.21.15.132:0/2186771212 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd16b8640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 0x7f2ccc19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2ccbfff640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc101930 0x7f2ccc196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 -- 172.21.15.132:0/2186771212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 msgr2=0x7f2ccc19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 0x7f2ccc19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 -- 172.21.15.132:0/2186771212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc101930 msgr2=0x7f2ccc196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc101930 0x7f2ccc196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 -- 172.21.15.132:0/2186771212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19e630 con 0x7f2ccc100740 2024-03-18T23:06:46.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd0eb7640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000e9c0 tx=0x7f2cc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000cd80 con 0x7f2ccc100740 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19e920 con 0x7f2ccc100740 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc00035f0 con 0x7f2ccc100740 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0010690 con 0x7f2ccc100740 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.817+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc19ee90 con 0x7f2ccc100740 2024-03-18T23:06:46.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.819+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc107a60 con 0x7f2ccc100740 2024-03-18T23:06:46.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.819+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f2cc00037a0 con 0x7f2ccc100740 2024-03-18T23:06:46.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.820+0000 7f2cc9ffb640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 0x7f2cbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:46.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.820+0000 7f2ccbfff640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 0x7f2cbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:46.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.820+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f2cc00ce970 con 0x7f2ccc100740 2024-03-18T23:06:46.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.820+0000 7f2ccbfff640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 0x7f2cbc0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c7b0 tx=0x7f2cb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:46.827 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:46.827+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2cc009beb0 con 0x7f2ccc100740 2024-03-18T23:06:47.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.044+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ccc062420 con 0x7f2ccc100740 2024-03-18T23:06:47.045 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.045+0000 7f2cc9ffb640 1 -- 172.21.15.132:0/2186771212 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f2cc009b7b0 con 0x7f2ccc100740 2024-03-18T23:06:47.046 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:47.046 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-18T23:06:47.046 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 4 2024-03-18T23:06:47.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 msgr2=0x7f2cbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:47.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 0x7f2cbc0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c7b0 tx=0x7f2cb8005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:47.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 msgr2=0x7f2ccc196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:47.048 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000e9c0 tx=0x7f2cc000ee80 comp rx=0 tx=0).stop 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 shutdown_connections 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2cbc0a2800 0x7f2cbc0a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ccc1022f0 0x7f2ccc19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ccc101930 0x7f2ccc196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 --2- 172.21.15.132:0/2186771212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ccc100740 0x7f2ccc196a10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 >> 172.21.15.132:0/2186771212 conn(0x7f2ccc0fbcb0 msgr2=0x7f2ccc104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 shutdown_connections 2024-03-18T23:06:47.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:47.047+0000 7f2cd311b640 1 -- 172.21.15.132:0/2186771212 wait complete. 2024-03-18T23:06:47.272 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:47 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2186771212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:47.272 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:06:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d[72666]: starting mds.d at 2024-03-18T23:06:47.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:47 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2186771212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:47.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:47 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2186771212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:47.615 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:06:47 smithi132 bash[72649]: 30ccf165fb8dbb062e4c74ddf56f6a5866696681d7f594655573603799afdd7b 2024-03-18T23:06:47.615 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:06:47 smithi132 systemd[1]: Started Ceph mds.d for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:48.221 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:48.221 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.221 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.221 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.221 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:48.222 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:48.222 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:48 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:48.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:48.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:48.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:48 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:48.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:48 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:49.060 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- 172.21.15.132:0/213306380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 msgr2=0x7f714c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 --2- 172.21.15.132:0/213306380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c102100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7140009900 tx=0x7f714002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- 172.21.15.132:0/213306380 shutdown_connections 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 --2- 172.21.15.132:0/213306380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 0x7f714c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 --2- 172.21.15.132:0/213306380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 0x7f714c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 --2- 172.21.15.132:0/213306380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c102100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- 172.21.15.132:0/213306380 >> 172.21.15.132:0/213306380 conn(0x7f714c0fb400 msgr2=0x7f714c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- 172.21.15.132:0/213306380 shutdown_connections 2024-03-18T23:06:49.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- 172.21.15.132:0/213306380 wait complete. 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 Processor -- start 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.177+0000 7f7152f74640 1 -- start start 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 0x7f714c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 0x7f714c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10c3b0 con 0x7f714c101ce0 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10c230 con 0x7f714c100a10 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f714c10c530 con 0x7f714c1013d0 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:49.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57100/0 (socket says 172.21.15.132:57100) 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 -- 172.21.15.132:0/1496622029 learned_addr learned my addr 172.21.15.132:0/1496622029 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f714bfff640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 0x7f714c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7150d10640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 0x7f714c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 -- 172.21.15.132:0/1496622029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 msgr2=0x7f714c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 0x7f714c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 -- 172.21.15.132:0/1496622029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 msgr2=0x7f714c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 0x7f714c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 -- 172.21.15.132:0/1496622029 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c1a1960 con 0x7f714c101ce0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7151511640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f714400cce0 tx=0x7f714400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714400f450 con 0x7f714c101ce0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714400fad0 con 0x7f714c101ce0 2024-03-18T23:06:49.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c1a1c50 con 0x7f714c101ce0 2024-03-18T23:06:49.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.178+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144018c10 con 0x7f714c101ce0 2024-03-18T23:06:49.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.179+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a2150 con 0x7f714c101ce0 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.180+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c1a2380 con 0x7f714c101ce0 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.180+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f714400f5f0 con 0x7f714c101ce0 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.181+0000 7f7149ffb640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 0x7f713c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.181+0000 7f7150d10640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 0x7f713c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.181+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f71440cecb0 con 0x7f714c101ce0 2024-03-18T23:06:49.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.181+0000 7f7150d10640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 0x7f713c0a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f714002c7b0 tx=0x7f7140005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:49.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.186+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7144014030 con 0x7f714c101ce0 2024-03-18T23:06:49.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:49 smithi132 ceph-mon[32020]: Deploying daemon mds.e on smithi144 2024-03-18T23:06:49.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:49 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.132:6836/2263462076,v1:172.21.15.132:6837/2263462076] up:boot 2024-03-18T23:06:49.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:49 smithi132 ceph-mon[32020]: fsmap 4 up:standby 2024-03-18T23:06:49.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:49 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:06:49.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.398+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f714c1a2550 con 0x7f714c101ce0 2024-03-18T23:06:49.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.398+0000 7f7149ffb640 1 -- 172.21.15.132:0/1496622029 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7f714409c1f0 con 0x7f714c101ce0 2024-03-18T23:06:49.399 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:49.399 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-18T23:06:49.399 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 5 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 msgr2=0x7f713c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 0x7f713c0a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f714002c7b0 tx=0x7f7140005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 msgr2=0x7f714c1a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f714400cce0 tx=0x7f714400a620 comp rx=0 tx=0).stop 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 shutdown_connections 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f713c0a2800 0x7f713c0a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f714c101ce0 0x7f714c1a12f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f714c1013d0 0x7f714c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 --2- 172.21.15.132:0/1496622029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714c100a10 0x7f714c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:49.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 >> 172.21.15.132:0/1496622029 conn(0x7f714c0fb400 msgr2=0x7f714c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:49.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 shutdown_connections 2024-03-18T23:06:49.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:49.400+0000 7f7152f74640 1 -- 172.21.15.132:0/1496622029 wait complete. 2024-03-18T23:06:49.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:49 smithi196 ceph-mon[67089]: Deploying daemon mds.e on smithi144 2024-03-18T23:06:49.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:49 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.132:6836/2263462076,v1:172.21.15.132:6837/2263462076] up:boot 2024-03-18T23:06:49.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:49 smithi196 ceph-mon[67089]: fsmap 4 up:standby 2024-03-18T23:06:49.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:49 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:06:49.540 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-mon[66781]: Deploying daemon mds.e on smithi144 2024-03-18T23:06:49.540 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.132:6836/2263462076,v1:172.21.15.132:6837/2263462076] up:boot 2024-03-18T23:06:49.540 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-mon[66781]: fsmap 4 up:standby 2024-03-18T23:06:49.540 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:06:49.540 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 systemd[1]: Starting Ceph mds.e for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:49.835 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 podman[97485]: 2024-03-18 23:06:49.539960543 +0000 UTC m=+0.102647992 container create 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-18T23:06:49.835 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 podman[97485]: 2024-03-18 23:06:49.454822436 +0000 UTC m=+0.017509886 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:49.835 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 podman[97485]: 2024-03-18 23:06:49.750992023 +0000 UTC m=+0.313679456 container init 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T23:06:49.835 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 podman[97485]: 2024-03-18 23:06:49.753580693 +0000 UTC m=+0.316268125 container start 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-18T23:06:49.835 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-mds[97504]: -- 172.21.15.144:0/2909709802 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c130d0ba40 con 0x55c130cde800 2024-03-18T23:06:49.836 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: starting mds.e at 2024-03-18T23:06:49.836 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 bash[97485]: 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 2024-03-18T23:06:50.109 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:06:49 smithi144 systemd[1]: Started Ceph mds.e for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1496622029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:50.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:50.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:50 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1496622029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:50.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:50.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:50 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1496622029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:50.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:50 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:51.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:51 smithi132 ceph-mon[32020]: Deploying daemon mds.f on smithi196 2024-03-18T23:06:51.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:51 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6834/769602526,v1:172.21.15.144:6835/769602526] up:boot 2024-03-18T23:06:51.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:51 smithi132 ceph-mon[32020]: fsmap 5 up:standby 2024-03-18T23:06:51.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:51 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:06:51.413 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:51.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:51 smithi196 ceph-mon[67089]: Deploying daemon mds.f on smithi196 2024-03-18T23:06:51.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:51 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6834/769602526,v1:172.21.15.144:6835/769602526] up:boot 2024-03-18T23:06:51.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:51 smithi196 ceph-mon[67089]: fsmap 5 up:standby 2024-03-18T23:06:51.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:51 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:06:51.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.525+0000 7fb1977a9640 1 -- 172.21.15.132:0/2565215558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb190101930 msgr2=0x7fb190101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.525+0000 7fb1977a9640 1 --2- 172.21.15.132:0/2565215558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb190101930 0x7fb190101db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:51.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- 172.21.15.132:0/2565215558 shutdown_connections 2024-03-18T23:06:51.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- 172.21.15.132:0/2565215558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1901022f0 0x7fb190108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- 172.21.15.132:0/2565215558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb190101930 0x7fb190101db0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- 172.21.15.132:0/2565215558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 0x7fb190100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- 172.21.15.132:0/2565215558 >> 172.21.15.132:0/2565215558 conn(0x7fb1900fbcb0 msgr2=0x7fb1900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- 172.21.15.132:0/2565215558 shutdown_connections 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- 172.21.15.132:0/2565215558 wait complete. 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 Processor -- start 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- start start 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 0x7fb1900694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb190101930 0x7fb19019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010c3c0 con 0x7fb1901022f0 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010c240 con 0x7fb190101930 2024-03-18T23:06:51.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.526+0000 7fb1977a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010c540 con 0x7fb190100740 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57124/0 (socket says 172.21.15.132:57124) 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195545640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 0x7fb1900694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 -- 172.21.15.132:0/627935028 learned_addr learned my addr 172.21.15.132:0/627935028 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb194d44640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb190101930 0x7fb19019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 -- 172.21.15.132:0/627935028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 msgr2=0x7fb1900694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 0x7fb1900694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 -- 172.21.15.132:0/627935028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb190101930 msgr2=0x7fb19019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb190101930 0x7fb19019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 -- 172.21.15.132:0/627935028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019f810 con 0x7fb1901022f0 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb195d46640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cad0 tx=0x7fb18800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:51.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18800ae20 con 0x7fb1901022f0 2024-03-18T23:06:51.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188005020 con 0x7fb1901022f0 2024-03-18T23:06:51.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19019faa0 con 0x7fb1901022f0 2024-03-18T23:06:51.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18800f6f0 con 0x7fb1901022f0 2024-03-18T23:06:51.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.527+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19019ff70 con 0x7fb1901022f0 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.528+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190107ae0 con 0x7fb1901022f0 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.529+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb1880051d0 con 0x7fb1901022f0 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.530+0000 7fb1867fc640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 0x7fb17c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.530+0000 7fb195545640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 0x7fb17c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.530+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb1880ce630 con 0x7fb1901022f0 2024-03-18T23:06:51.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.530+0000 7fb195545640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 0x7fb17c0a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb190101790 tx=0x7fb18c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:51.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.534+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb188014030 con 0x7fb1901022f0 2024-03-18T23:06:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:51 smithi144 ceph-mon[66781]: Deploying daemon mds.f on smithi196 2024-03-18T23:06:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:51 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6834/769602526,v1:172.21.15.144:6835/769602526] up:boot 2024-03-18T23:06:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:51 smithi144 ceph-mon[66781]: fsmap 5 up:standby 2024-03-18T23:06:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:51 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:06:51.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.749+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb19002d3c0 con 0x7fb1901022f0 2024-03-18T23:06:51.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.749+0000 7fb1867fc640 1 -- 172.21.15.132:0/627935028 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7fb18809bb70 con 0x7fb1901022f0 2024-03-18T23:06:51.750 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:51.750 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-18T23:06:51.750 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 6 2024-03-18T23:06:51.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 msgr2=0x7fb17c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 0x7fb17c0a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb190101790 tx=0x7fb18c005f90 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 msgr2=0x7fb19019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cad0 tx=0x7fb18800cf90 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 shutdown_connections 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb17c0a2800 0x7fb17c0a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1901022f0 0x7fb19019ef00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb190101930 0x7fb19019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 --2- 172.21.15.132:0/627935028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb190100740 0x7fb1900694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:51.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 >> 172.21.15.132:0/627935028 conn(0x7fb1900fbcb0 msgr2=0x7fb190104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:51.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 shutdown_connections 2024-03-18T23:06:51.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:51.751+0000 7fb1977a9640 1 -- 172.21.15.132:0/627935028 wait complete. 2024-03-18T23:06:52.148 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:51 smithi196 systemd[1]: Starting Ceph mds.f for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:52.410 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:52 smithi196 ceph-mon[67089]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:52.410 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:52 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/627935028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:52.411 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 podman[97390]: 2024-03-18 23:06:52.062628159 +0000 UTC m=+0.017729241 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:52.411 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 podman[97390]: 2024-03-18 23:06:52.166765168 +0000 UTC m=+0.121866242 container create 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T23:06:52.411 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 podman[97390]: 2024-03-18 23:06:52.385604687 +0000 UTC m=+0.340705773 container init 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-18T23:06:52.411 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 podman[97390]: 2024-03-18 23:06:52.388112547 +0000 UTC m=+0.343213626 container start 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-18T23:06:52.411 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 ceph-mds[97409]: -- 172.21.15.196:0/2177661265 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0315c1a40 con 0x55b031594800 2024-03-18T23:06:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:52 smithi144 ceph-mon[66781]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:52 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/627935028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:52.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:52 smithi132 ceph-mon[32020]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:52.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:52 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/627935028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:52.723 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: starting mds.f at 2024-03-18T23:06:52.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 bash[97390]: 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a 2024-03-18T23:06:52.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:06:52 smithi196 systemd[1]: Started Ceph mds.f for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:53.764 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:53.792 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: Deploying daemon mds.c on smithi196 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6834/559973533,v1:172.21.15.196:6835/559973533] up:boot 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: fsmap 6 up:standby 2024-03-18T23:06:53.793 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:53 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:06:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: Deploying daemon mds.c on smithi196 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6834/559973533,v1:172.21.15.196:6835/559973533] up:boot 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: fsmap 6 up:standby 2024-03-18T23:06:53.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:53 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 -- 172.21.15.132:0/1184988615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34102a30 msgr2=0x7f1f34102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 --2- 172.21.15.132:0/1184988615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34102a30 0x7f1f34102eb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009900 tx=0x7f1f2802bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 -- 172.21.15.132:0/1184988615 shutdown_connections 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 --2- 172.21.15.132:0/1184988615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f34109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 --2- 172.21.15.132:0/1184988615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34102a30 0x7f1f34102eb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 --2- 172.21.15.132:0/1184988615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34101840 0x7f1f34101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 -- 172.21.15.132:0/1184988615 >> 172.21.15.132:0/1184988615 conn(0x7f1f340fcdd0 msgr2=0x7f1f340ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 -- 172.21.15.132:0/1184988615 shutdown_connections 2024-03-18T23:06:53.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.876+0000 7f1f38960640 1 -- 172.21.15.132:0/1184988615 wait complete. 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 Processor -- start 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 -- start start 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34102a30 0x7f1f341980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f3419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410d4c0 con 0x7f1f34101840 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410d340 con 0x7f1f341033f0 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f38960640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3410d640 con 0x7f1f34102a30 2024-03-18T23:06:53.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58522/0 (socket says 172.21.15.132:58522) 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 -- 172.21.15.132:0/3127798281 learned_addr learned my addr 172.21.15.132:0/3127798281 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f31d9b640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34102a30 0x7f1f341980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f32d9d640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f3419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 -- 172.21.15.132:0/3127798281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34102a30 msgr2=0x7f1f341980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34102a30 0x7f1f341980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 -- 172.21.15.132:0/3127798281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 msgr2=0x7f1f3419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f3419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 -- 172.21.15.132:0/3127798281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3419f780 con 0x7f1f34101840 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f32d9d640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f3419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.877+0000 7f1f3259c640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d950 tx=0x7f1f2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.878+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c017070 con 0x7f1f34101840 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.878+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2c009c30 con 0x7f1f34101840 2024-03-18T23:06:53.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.878+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c00f600 con 0x7f1f34101840 2024-03-18T23:06:53.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.878+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3419fa70 con 0x7f1f34101840 2024-03-18T23:06:53.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.878+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3419ff90 con 0x7f1f34101840 2024-03-18T23:06:53.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.879+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f34108b60 con 0x7f1f34101840 2024-03-18T23:06:53.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.880+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f1f2c01a070 con 0x7f1f34101840 2024-03-18T23:06:53.882 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.880+0000 7f1f237fe640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 0x7f1f1c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:53.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.881+0000 7f1f31d9b640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 0x7f1f1c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:53.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.881+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1f2c0cdf70 con 0x7f1f34101840 2024-03-18T23:06:53.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.881+0000 7f1f31d9b640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 0x7f1f1c0a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f2802c7b0 tx=0x7f1f28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:53.885 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:53.885+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f2c009de0 con 0x7f1f34101840 2024-03-18T23:06:53.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:53.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:53.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: Deploying daemon mds.c on smithi196 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6834/559973533,v1:172.21.15.196:6835/559973533] up:boot 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: fsmap 6 up:standby 2024-03-18T23:06:53.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:53 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:06:54.100 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.099+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f3402d1f0 con 0x7f1f34101840 2024-03-18T23:06:54.101 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.100+0000 7f1f237fe640 1 -- 172.21.15.132:0/3127798281 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f1f2c09b4b0 con 0x7f1f34101840 2024-03-18T23:06:54.101 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:54.102 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-18T23:06:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 7 2024-03-18T23:06:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 msgr2=0x7f1f1c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:54.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 0x7f1f1c0a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f2802c7b0 tx=0x7f1f28005fd0 comp rx=0 tx=0).stop 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 msgr2=0x7f1f34197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d950 tx=0x7f1f2c00de10 comp rx=0 tx=0).stop 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 shutdown_connections 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1f1c0a2880 0x7f1f1c0a4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f341033f0 0x7f1f3419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:54.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f34102a30 0x7f1f341980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 --2- 172.21.15.132:0/3127798281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f34101840 0x7f1f34197b60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 >> 172.21.15.132:0/3127798281 conn(0x7f1f340fcdd0 msgr2=0x7f1f34105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 shutdown_connections 2024-03-18T23:06:54.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:54.102+0000 7f1f38960640 1 -- 172.21.15.132:0/3127798281 wait complete. 2024-03-18T23:06:54.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:54 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3127798281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:54.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:54 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3127798281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:54.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:54 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3127798281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:54.972 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:54 smithi196 systemd[1]: Starting Ceph mds.c for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:55.224 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:54 smithi196 podman[97701]: 2024-03-18 23:06:54.874561065 +0000 UTC m=+0.018208614 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:55.224 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:54 smithi196 podman[97701]: 2024-03-18 23:06:54.99439938 +0000 UTC m=+0.138046930 container create 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 podman[97701]: 2024-03-18 23:06:55.230565268 +0000 UTC m=+0.374212823 container init 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 podman[97701]: 2024-03-18 23:06:55.233161463 +0000 UTC m=+0.376809026 container start 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mds[97720]: -- 172.21.15.196:0/1586652351 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad7fb27a40 con 0x55ad7fafa800 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: starting mds.c at 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 bash[97701]: 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 2024-03-18T23:06:55.571 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:06:55 smithi196 systemd[1]: Started Ceph mds.c for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:55 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:55.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:55 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:55.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:55 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:56.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:56.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 -- 172.21.15.132:0/153397719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430101220 msgr2=0x7f4430101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 --2- 172.21.15.132:0/153397719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430101220 0x7f4430101aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4428009ca0 tx=0x7f442802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 -- 172.21.15.132:0/153397719 shutdown_connections 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 --2- 172.21.15.132:0/153397719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430101220 0x7f4430101aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 --2- 172.21.15.132:0/153397719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4430100860 0x7f4430100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 --2- 172.21.15.132:0/153397719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44300ffb00 0x7f44300fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 -- 172.21.15.132:0/153397719 >> 172.21.15.132:0/153397719 conn(0x7f44300fb450 msgr2=0x7f44300fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 -- 172.21.15.132:0/153397719 shutdown_connections 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.227+0000 7f443764b640 1 -- 172.21.15.132:0/153397719 wait complete. 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 Processor -- start 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 -- start start 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44300ffb00 0x7f4430199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4430101220 0x7f44301a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f443010f6e0 con 0x7f4430100860 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f443010f560 con 0x7f44300ffb00 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f443764b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f443010f860 con 0x7f4430101220 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f4434be6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f4434be6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58542/0 (socket says 172.21.15.132:58542) 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f4434be6640 1 -- 172.21.15.132:0/2910583894 learned_addr learned my addr 172.21.15.132:0/2910583894 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f44353e7640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44300ffb00 0x7f4430199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.228+0000 7f4435be8640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4430101220 0x7f44301a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 -- 172.21.15.132:0/2910583894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4430101220 msgr2=0x7f44301a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4430101220 0x7f44301a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 -- 172.21.15.132:0/2910583894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44300ffb00 msgr2=0x7f4430199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44300ffb00 0x7f4430199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 -- 172.21.15.132:0/2910583894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44301a1920 con 0x7f4430100860 2024-03-18T23:06:56.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f4434be6640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9c0 tx=0x7f442c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00cde0 con 0x7f4430100860 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c010910 con 0x7f4430100860 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44301a1c10 con 0x7f4430100860 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c018a00 con 0x7f4430100860 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.229+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44301a2160 con 0x7f4430100860 2024-03-18T23:06:56.233 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.230+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443010b610 con 0x7f4430100860 2024-03-18T23:06:56.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.231+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f442c01f020 con 0x7f4430100860 2024-03-18T23:06:56.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.232+0000 7f44267fc640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 0x7f441c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:56.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.232+0000 7f44353e7640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 0x7f441c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:56.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.232+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f442c0cee00 con 0x7f4430100860 2024-03-18T23:06:56.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.232+0000 7f44353e7640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 0x7f441c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f443019b1e0 tx=0x7f442000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:56.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.235+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f442c09c340 con 0x7f4430100860 2024-03-18T23:06:56.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.448+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4430062420 con 0x7f4430100860 2024-03-18T23:06:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.448+0000 7f44267fc640 1 -- 172.21.15.132:0/2910583894 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f442c09bc40 con 0x7f4430100860 2024-03-18T23:06:56.449 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:56.449 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-18T23:06:56.449 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 8 2024-03-18T23:06:56.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 msgr2=0x7f441c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 0x7f441c0a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f443019b1e0 tx=0x7f442000b390 comp rx=0 tx=0).stop 2024-03-18T23:06:56.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 msgr2=0x7f443019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:56.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f442c00e9c0 tx=0x7f442c00ee80 comp rx=0 tx=0).stop 2024-03-18T23:06:56.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 shutdown_connections 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f441c0a2800 0x7f441c0a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4430101220 0x7f44301a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4430100860 0x7f443019a250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 --2- 172.21.15.132:0/2910583894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44300ffb00 0x7f4430199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.450+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 >> 172.21.15.132:0/2910583894 conn(0x7f44300fb450 msgr2=0x7f44300fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.451+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 shutdown_connections 2024-03-18T23:06:56.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:56.451+0000 7f443764b640 1 -- 172.21.15.132:0/2910583894 wait complete. 2024-03-18T23:06:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:56 smithi144 ceph-mon[66781]: Deploying daemon mds.h on smithi144 2024-03-18T23:06:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:56 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6836/3483317240,v1:172.21.15.196:6837/3483317240] up:boot 2024-03-18T23:06:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:56 smithi144 ceph-mon[66781]: fsmap 7 up:standby 2024-03-18T23:06:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:56 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:06:56.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:56 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2910583894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:56.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:56 smithi132 ceph-mon[32020]: Deploying daemon mds.h on smithi144 2024-03-18T23:06:56.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:56 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6836/3483317240,v1:172.21.15.196:6837/3483317240] up:boot 2024-03-18T23:06:56.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:56 smithi132 ceph-mon[32020]: fsmap 7 up:standby 2024-03-18T23:06:56.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:56 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:06:56.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:56 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2910583894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:56 smithi196 ceph-mon[67089]: Deploying daemon mds.h on smithi144 2024-03-18T23:06:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:56 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6836/3483317240,v1:172.21.15.196:6837/3483317240] up:boot 2024-03-18T23:06:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:56 smithi196 ceph-mon[67089]: fsmap 7 up:standby 2024-03-18T23:06:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:56 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:06:56.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:56 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2910583894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:57.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:57 smithi144 ceph-mon[66781]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:57.844 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:57 smithi144 systemd[1]: Starting Ceph mds.h for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:06:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:57 smithi132 ceph-mon[32020]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:57.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:57 smithi196 ceph-mon[67089]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:58.109 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:57 smithi144 podman[97797]: 2024-03-18 23:06:57.745933657 +0000 UTC m=+0.017765185 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:06:58.109 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:57 smithi144 podman[97797]: 2024-03-18 23:06:57.8434785 +0000 UTC m=+0.115310017 container create 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:06:58.109 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 podman[97797]: 2024-03-18 23:06:58.054770063 +0000 UTC m=+0.326601567 container init 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T23:06:58.109 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 podman[97797]: 2024-03-18 23:06:58.057421066 +0000 UTC m=+0.329252584 container start 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-18T23:06:58.109 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 ceph-mds[97817]: -- 172.21.15.144:0/27174824 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc0758fa40 con 0x55fc07562800 2024-03-18T23:06:58.110 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: starting mds.h at 2024-03-18T23:06:58.463 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 -- 172.21.15.132:0/4001525591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 msgr2=0x7f076c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 --2- 172.21.15.132:0/4001525591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c100b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0760009900 tx=0x7f076002bc30 comp rx=0 tx=0).stop 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 -- 172.21.15.132:0/4001525591 shutdown_connections 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 --2- 172.21.15.132:0/4001525591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 0x7f076c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 --2- 172.21.15.132:0/4001525591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 0x7f076c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 --2- 172.21.15.132:0/4001525591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c100b50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 -- 172.21.15.132:0/4001525591 >> 172.21.15.132:0/4001525591 conn(0x7f076c0fbcc0 msgr2=0x7f076c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 -- 172.21.15.132:0/4001525591 shutdown_connections 2024-03-18T23:06:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.575+0000 7f0772ba9640 1 -- 172.21.15.132:0/4001525591 wait complete. 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 Processor -- start 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 -- start start 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 0x7f076c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 0x7f076c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f076c10c420 con 0x7f076c100730 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f076c10c2a0 con 0x7f076c1022e0 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0772ba9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f076c10c5a0 con 0x7f076c101920 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:58.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58548/0 (socket says 172.21.15.132:58548) 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 -- 172.21.15.132:0/938030454 learned_addr learned my addr 172.21.15.132:0/938030454 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f076bfff640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 0x7f076c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0771146640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 0x7f076c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 -- 172.21.15.132:0/938030454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 msgr2=0x7f076c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 0x7f076c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 -- 172.21.15.132:0/938030454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 msgr2=0x7f076c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 0x7f076c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.576+0000 7f0770945640 1 -- 172.21.15.132:0/938030454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c19f880 con 0x7f076c100730 2024-03-18T23:06:58.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.577+0000 7f0770945640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0760005ab0 tx=0x7f0760005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:58.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.579+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760041070 con 0x7f076c100730 2024-03-18T23:06:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.579+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c19fb10 con 0x7f076c100730 2024-03-18T23:06:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.580+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c1a0050 con 0x7f076c100730 2024-03-18T23:06:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.580+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0760004c80 con 0x7f076c100730 2024-03-18T23:06:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.580+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076004b060 con 0x7f076c100730 2024-03-18T23:06:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.581+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07500052c0 con 0x7f076c100730 2024-03-18T23:06:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.582+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0760044070 con 0x7f076c100730 2024-03-18T23:06:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.583+0000 7f0769ffb640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 0x7f075c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:06:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.583+0000 7f076bfff640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 0x7f075c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:06:58.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.583+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f07600f03b0 con 0x7f076c100730 2024-03-18T23:06:58.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.583+0000 7f076bfff640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 0x7f075c0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0758005fd0 tx=0x7f075800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:06:58.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.586+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07600bd2a0 con 0x7f076c100730 2024-03-18T23:06:58.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 bash[97797]: 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e 2024-03-18T23:06:58.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:06:58 smithi144 systemd[1]: Started Ceph mds.h for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:06:58.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.796+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07500055b0 con 0x7f076c100730 2024-03-18T23:06:58.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.797+0000 7f0769ffb640 1 -- 172.21.15.132:0/938030454 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f07600c1340 con 0x7f076c100730 2024-03-18T23:06:58.798 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:06:58.798 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-18T23:06:58.798 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 9 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 msgr2=0x7f075c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 0x7f075c0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0758005fd0 tx=0x7f075800b390 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 msgr2=0x7f076c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0760005ab0 tx=0x7f0760005dc0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 shutdown_connections 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f075c0a2800 0x7f075c0a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1022e0 0x7f076c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f076c101920 0x7f076c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 --2- 172.21.15.132:0/938030454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f076c100730 0x7f076c197c70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 >> 172.21.15.132:0/938030454 conn(0x7f076c0fbcc0 msgr2=0x7f076c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 shutdown_connections 2024-03-18T23:06:58.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:06:58.799+0000 7f0772ba9640 1 -- 172.21.15.132:0/938030454 wait complete. 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:59.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: Deploying daemon mds.i on smithi196 2024-03-18T23:06:59.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:boot 2024-03-18T23:06:59.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: fsmap 8 up:standby 2024-03-18T23:06:59.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:06:59.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/938030454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:59.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:06:59 smithi196 ceph-mon[67089]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: Deploying daemon mds.i on smithi196 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:boot 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: fsmap 8 up:standby 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:06:59.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/938030454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:59.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:06:59 smithi144 ceph-mon[66781]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: Deploying daemon mds.i on smithi196 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:boot 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: fsmap 8 up:standby 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/938030454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:06:59.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:06:59 smithi132 ceph-mon[32020]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:00.474 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 systemd[1]: Starting Ceph mds.i for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:07:00.812 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:00.849 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 podman[98007]: 2024-03-18 23:07:00.495724678 +0000 UTC m=+0.018149406 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:07:00.849 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 podman[98007]: 2024-03-18 23:07:00.596942962 +0000 UTC m=+0.119367689 container create 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T23:07:00.849 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 podman[98007]: 2024-03-18 23:07:00.849190608 +0000 UTC m=+0.371615354 container init 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:07:00.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 -- 172.21.15.132:0/3657947265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0100760 msgr2=0x7ff1f0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:00.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/3657947265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0100760 0x7ff1f0100b80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009900 tx=0x7ff1e002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 -- 172.21.15.132:0/3657947265 shutdown_connections 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/3657947265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0102310 0x7ff1f0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/3657947265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0101950 0x7ff1f0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/3657947265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0100760 0x7ff1f0100b80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 -- 172.21.15.132:0/3657947265 >> 172.21.15.132:0/3657947265 conn(0x7ff1f00fbcd0 msgr2=0x7ff1f00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 -- 172.21.15.132:0/3657947265 shutdown_connections 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.925+0000 7ff1f51b2640 1 -- 172.21.15.132:0/3657947265 wait complete. 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 Processor -- start 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 -- start start 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0100760 0x7ff1f006a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 0x7ff1f019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:00.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f010c3b0 con 0x7ff1f0102310 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f010c230 con 0x7ff1f0101950 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1f51b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f010c530 con 0x7ff1f0100760 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1ef59e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58562/0 (socket says 172.21.15.132:58562) 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1ef59e640 1 -- 172.21.15.132:0/2644744113 learned_addr learned my addr 172.21.15.132:0/2644744113 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1eed9d640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0100760 0x7ff1f006a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.926+0000 7ff1e7fff640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 0x7ff1f019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 -- 172.21.15.132:0/2644744113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0100760 msgr2=0x7ff1f006a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0100760 0x7ff1f006a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 -- 172.21.15.132:0/2644744113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 msgr2=0x7ff1f019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:00.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 0x7ff1f019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 -- 172.21.15.132:0/2644744113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f019f800 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1e7fff640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 0x7ff1f019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ef59e640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800eb10 tx=0x7ff1e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8004b20 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f019fa90 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e8004cc0 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8010760 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.927+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f019ffe0 con 0x7ff1f0102310 2024-03-18T23:07:00.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.928+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0107a80 con 0x7ff1f0102310 2024-03-18T23:07:00.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.929+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7ff1e8005020 con 0x7ff1f0102310 2024-03-18T23:07:00.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.929+0000 7ff1ecd99640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 0x7ff1dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:00.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.930+0000 7ff1eed9d640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 0x7ff1dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:00.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.930+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff1e80cebe0 con 0x7ff1f0102310 2024-03-18T23:07:00.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.930+0000 7ff1eed9d640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 0x7ff1dc0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0000c00 tx=0x7ff1e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:00.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:00.934+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1e809c120 con 0x7ff1f0102310 2024-03-18T23:07:01.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.157+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1f002d3f0 con 0x7ff1f0102310 2024-03-18T23:07:01.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.158+0000 7ff1ecd99640 1 -- 172.21.15.132:0/2644744113 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7ff1e809ba20 con 0x7ff1f0102310 2024-03-18T23:07:01.159 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:01.159 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:07:01.159 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 10 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 msgr2=0x7ff1dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 0x7ff1dc0a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0000c00 tx=0x7ff1e0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 msgr2=0x7ff1f019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800eb10 tx=0x7ff1e800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 shutdown_connections 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1dc0a2800 0x7ff1dc0a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1f0102310 0x7ff1f019eef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f0101950 0x7ff1f019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 --2- 172.21.15.132:0/2644744113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1f0100760 0x7ff1f006a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 >> 172.21.15.132:0/2644744113 conn(0x7ff1f00fbcd0 msgr2=0x7ff1f0104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:01.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 shutdown_connections 2024-03-18T23:07:01.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:01.160+0000 7ff1f51b2640 1 -- 172.21.15.132:0/2644744113 wait complete. 2024-03-18T23:07:01.224 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 podman[98007]: 2024-03-18 23:07:00.851835847 +0000 UTC m=+0.374260575 container start 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:07:01.224 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 ceph-mds[98026]: -- 172.21.15.196:0/3135484544 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b101155a40 con 0x55b101128800 2024-03-18T23:07:01.224 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: starting mds.i at 2024-03-18T23:07:01.224 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 bash[98007]: 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 2024-03-18T23:07:01.224 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:07:00 smithi196 systemd[1]: Started Ceph mds.i for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:boot 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: fsmap 9 up:standby 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:02.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: Deploying daemon mds.g on smithi132 2024-03-18T23:07:02.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:02.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2644744113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:boot 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: fsmap 9 up:standby 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: Deploying daemon mds.g on smithi132 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:02.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2644744113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:boot 2024-03-18T23:07:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: fsmap 9 up:standby 2024-03-18T23:07:02.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: Deploying daemon mds.g on smithi132 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:02.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2644744113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:03.173 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:03.202 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 systemd[1]: Starting Ceph mds.g for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:07:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 -- 172.21.15.132:0/3674475092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 msgr2=0x7fef1c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3674475092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c102ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef10009900 tx=0x7fef1002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 -- 172.21.15.132:0/3674475092 shutdown_connections 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3674475092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3674475092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c102ec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3674475092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 0x7fef1c101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 -- 172.21.15.132:0/3674475092 >> 172.21.15.132:0/3674475092 conn(0x7fef1c0fce00 msgr2=0x7fef1c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 -- 172.21.15.132:0/3674475092 shutdown_connections 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.282+0000 7fef21c3d640 1 -- 172.21.15.132:0/3674475092 wait complete. 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 Processor -- start 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 -- start start 2024-03-18T23:07:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 0x7fef1c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c10d5c0 con 0x7fef1c102a40 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c10d440 con 0x7fef1c103400 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef21c3d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c10d740 con 0x7fef1c101850 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef1affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef1affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40520/0 (socket says 172.21.15.132:40520) 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.283+0000 7fef1affd640 1 -- 172.21.15.132:0/3267328497 learned_addr learned my addr 172.21.15.132:0/3267328497 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1b7fe640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 0x7fef1c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1bfff640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 -- 172.21.15.132:0/3267328497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 msgr2=0x7fef1c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 0x7fef1c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 -- 172.21.15.132:0/3267328497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 msgr2=0x7fef1c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 -- 172.21.15.132:0/3267328497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c19f860 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1bfff640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef1affd640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef10009230 tx=0x7fef10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10041070 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef100054e0 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c19faf0 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef100346a0 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.284+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c19ffe0 con 0x7fef1c102a40 2024-03-18T23:07:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.285+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1c108b70 con 0x7fef1c102a40 2024-03-18T23:07:03.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.286+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fef10044070 con 0x7fef1c102a40 2024-03-18T23:07:03.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.286+0000 7fef18ff9640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 0x7fef040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:03.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.287+0000 7fef1b7fe640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 0x7fef040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:03.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.287+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fef100f0560 con 0x7fef1c102a40 2024-03-18T23:07:03.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.287+0000 7fef1b7fe640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 0x7fef040a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef1c199120 tx=0x7fef0c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:03.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.291+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef100bdaa0 con 0x7fef1c102a40 2024-03-18T23:07:03.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.511+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef1c02d410 con 0x7fef1c102a40 2024-03-18T23:07:03.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.512+0000 7fef18ff9640 1 -- 172.21.15.132:0/3267328497 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7fef100bd3a0 con 0x7fef1c102a40 2024-03-18T23:07:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:03.513 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:07:03.513 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 10 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 msgr2=0x7fef040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 0x7fef040a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef1c199120 tx=0x7fef0c0094f0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 msgr2=0x7fef1c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef10009230 tx=0x7fef10004040 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 shutdown_connections 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef040a2800 0x7fef040a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c103400 0x7fef1c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef1c102a40 0x7fef1c198190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 --2- 172.21.15.132:0/3267328497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef1c101850 0x7fef1c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 >> 172.21.15.132:0/3267328497 conn(0x7fef1c0fce00 msgr2=0x7fef1c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:03.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 shutdown_connections 2024-03-18T23:07:03.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:03.514+0000 7fef21c3d640 1 -- 172.21.15.132:0/3267328497 wait complete. 2024-03-18T23:07:03.525 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 podman[73212]: 2024-03-18 23:07:03.206627977 +0000 UTC m=+0.019803444 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:07:03.525 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 podman[73212]: 2024-03-18 23:07:03.315716655 +0000 UTC m=+0.128892112 container create e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:07:03.865 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 podman[73212]: 2024-03-18 23:07:03.526617628 +0000 UTC m=+0.339793103 container init e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-18T23:07:03.866 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 podman[73212]: 2024-03-18 23:07:03.52936366 +0000 UTC m=+0.342539118 container start e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-18T23:07:03.866 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 ceph-mds[73266]: -- 172.21.15.132:0/3172360813 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6cb749a40 con 0x55a6cb71d000 2024-03-18T23:07:03.866 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g[73260]: starting mds.g at 2024-03-18T23:07:03.866 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 bash[73212]: e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a 2024-03-18T23:07:03.866 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:07:03 smithi132 systemd[1]: Started Ceph mds.g for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3267328497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:04.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:04 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3267328497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:04.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:04.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:04 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:04.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3267328497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:04 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:05.284 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:05 smithi132 ceph-mon[32020]: Deploying daemon mds.a on smithi132 2024-03-18T23:07:05.284 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:05 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.132:6838/4158656425,v1:172.21.15.132:6839/4158656425] up:boot 2024-03-18T23:07:05.284 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:05 smithi132 ceph-mon[32020]: fsmap 10 up:standby 2024-03-18T23:07:05.284 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:05 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:07:05.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:05 smithi144 ceph-mon[66781]: Deploying daemon mds.a on smithi132 2024-03-18T23:07:05.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:05 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.132:6838/4158656425,v1:172.21.15.132:6839/4158656425] up:boot 2024-03-18T23:07:05.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:05 smithi144 ceph-mon[66781]: fsmap 10 up:standby 2024-03-18T23:07:05.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:05 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:07:05.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:05 smithi196 ceph-mon[67089]: Deploying daemon mds.a on smithi132 2024-03-18T23:07:05.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:05 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.132:6838/4158656425,v1:172.21.15.132:6839/4158656425] up:boot 2024-03-18T23:07:05.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:05 smithi196 ceph-mon[67089]: fsmap 10 up:standby 2024-03-18T23:07:05.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:05 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:07:05.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:05.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 -- 172.21.15.132:0/2579691207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f7810b190 msgr2=0x7f0f7810d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 --2- 172.21.15.132:0/2579691207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f7810b190 0x7f0f7810d620 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009ca0 tx=0x7f0f7002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 -- 172.21.15.132:0/2579691207 shutdown_connections 2024-03-18T23:07:05.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 --2- 172.21.15.132:0/2579691207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810db60 0x7f0f7810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 --2- 172.21.15.132:0/2579691207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f7810b190 0x7f0f7810d620 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 --2- 172.21.15.132:0/2579691207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f78108820 0x7f0f7810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 -- 172.21.15.132:0/2579691207 >> 172.21.15.132:0/2579691207 conn(0x7f0f7806e690 msgr2=0x7f0f78070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:05.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 -- 172.21.15.132:0/2579691207 shutdown_connections 2024-03-18T23:07:05.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.668+0000 7f0f7e614640 1 -- 172.21.15.132:0/2579691207 wait complete. 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.669+0000 7f0f7e614640 1 Processor -- start 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.669+0000 7f0f7e614640 1 -- start start 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810b190 0x7f0f781a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7810db60 0x7f0f781adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7811c310 con 0x7f0f78108820 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7811c190 con 0x7f0f7810db60 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7e614640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7811c490 con 0x7f0f7810b190 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40570/0 (socket says 172.21.15.132:40570) 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 -- 172.21.15.132:0/252798583 learned_addr learned my addr 172.21.15.132:0/252798583 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f777fe640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810b190 0x7f0f781a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f7cbb1640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7810db60 0x7f0f781adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 -- 172.21.15.132:0/252798583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810b190 msgr2=0x7f0f781a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810b190 0x7f0f781a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 -- 172.21.15.132:0/252798583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7810db60 msgr2=0x7f0f781adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7810db60 0x7f0f781adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 -- 172.21.15.132:0/252798583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781ae5b0 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.670+0000 7f0f77fff640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e9c0 tx=0x7f0f6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.671+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800cd80 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.671+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f781ae8a0 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.671+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f680035f0 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.671+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68010690 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.671+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781aee10 con 0x7f0f78108820 2024-03-18T23:07:05.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.672+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f78071190 con 0x7f0f78108820 2024-03-18T23:07:05.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.673+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0f680037a0 con 0x7f0f78108820 2024-03-18T23:07:05.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.673+0000 7f0f757fa640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 0x7f0f600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:05.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.674+0000 7f0f777fe640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 0x7f0f600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:05.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.674+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0f680cea50 con 0x7f0f78108820 2024-03-18T23:07:05.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.674+0000 7f0f777fe640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 0x7f0f600a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f7002c700 tx=0x7f0f700323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:05.682 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.681+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f6809c020 con 0x7f0f78108820 2024-03-18T23:07:05.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.918+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f7802d1f0 con 0x7f0f78108820 2024-03-18T23:07:05.920 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.919+0000 7f0f757fa640 1 -- 172.21.15.132:0/252798583 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8845 (secure 0 0 0) 0x7f0f6809b920 con 0x7f0f78108820 2024-03-18T23:07:05.921 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:05.921 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:07:05.922 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 11 2024-03-18T23:07:05.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 msgr2=0x7f0f600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 0x7f0f600a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f7002c700 tx=0x7f0f700323c0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 msgr2=0x7f0f781a6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:05.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800e9c0 tx=0x7f0f6800ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:05.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 shutdown_connections 2024-03-18T23:07:05.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f600a2800 0x7f0f600a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7810db60 0x7f0f781adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7810b190 0x7f0f781a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 --2- 172.21.15.132:0/252798583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f78108820 0x7f0f781a6990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:05.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 >> 172.21.15.132:0/252798583 conn(0x7f0f7806e690 msgr2=0x7f0f78108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:05.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 shutdown_connections 2024-03-18T23:07:05.925 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:05.922+0000 7f0f7e614640 1 -- 172.21.15.132:0/252798583 wait complete. 2024-03-18T23:07:06.107 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 podman[73597]: 2024-03-18 23:07:05.951258436 +0000 UTC m=+0.023295466 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:07:06.357 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:06 smithi132 ceph-mon[32020]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:06.357 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:06 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/252798583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:06.357 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 podman[73597]: 2024-03-18 23:07:06.257380488 +0000 UTC m=+0.329417512 container init e1cfe9ae8e6f6a88c0f983fea7590d43be861c7e0274f80825623fa1e1a8a3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:07:06.358 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 podman[73597]: 2024-03-18 23:07:06.259983979 +0000 UTC m=+0.332020999 container start e1cfe9ae8e6f6a88c0f983fea7590d43be861c7e0274f80825623fa1e1a8a3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307) 2024-03-18T23:07:06.358 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 ceph-mds[73625]: -- 172.21.15.132:0/1963092034 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1dcf47a40 con 0x55f1dcf1b000 2024-03-18T23:07:06.358 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a[73621]: starting mds.a at 2024-03-18T23:07:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:06 smithi144 ceph-mon[66781]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:06.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:06 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/252798583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:06 smithi196 ceph-mon[67089]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:06.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:06 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/252798583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:06.615 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 bash[73597]: e1cfe9ae8e6f6a88c0f983fea7590d43be861c7e0274f80825623fa1e1a8a3a9 2024-03-18T23:07:06.615 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:07:06 smithi132 systemd[1]: Started Ceph mds.a for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:07:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: Deploying daemon mds.b on smithi144 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.132:6840/3314453651,v1:172.21.15.132:6841/3314453651] up:boot 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: fsmap 11 up:standby 2024-03-18T23:07:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:07 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:07.744 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:07.745 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: Deploying daemon mds.b on smithi144 2024-03-18T23:07:07.745 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:07.745 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.132:6840/3314453651,v1:172.21.15.132:6841/3314453651] up:boot 2024-03-18T23:07:07.745 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: fsmap 11 up:standby 2024-03-18T23:07:07.745 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:07 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T23:07:07.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:07.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: Deploying daemon mds.b on smithi144 2024-03-18T23:07:07.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:07.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.132:6840/3314453651,v1:172.21.15.132:6841/3314453651] up:boot 2024-03-18T23:07:07.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: fsmap 11 up:standby 2024-03-18T23:07:07.867 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:07 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:07:07.937 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 -- 172.21.15.132:0/503352338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0101930 msgr2=0x7f07d0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 --2- 172.21.15.132:0/503352338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0101930 0x7f07d0101db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 -- 172.21.15.132:0/503352338 shutdown_connections 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 --2- 172.21.15.132:0/503352338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d01022f0 0x7f07d0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 --2- 172.21.15.132:0/503352338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0101930 0x7f07d0101db0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 --2- 172.21.15.132:0/503352338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d0100740 0x7f07d0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 -- 172.21.15.132:0/503352338 >> 172.21.15.132:0/503352338 conn(0x7f07d00fbcd0 msgr2=0x7f07d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:08.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 -- 172.21.15.132:0/503352338 shutdown_connections 2024-03-18T23:07:08.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.053+0000 7f07d7244640 1 -- 172.21.15.132:0/503352338 wait complete. 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 Processor -- start 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 -- start start 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d0101930 0x7f07d010aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d01022f0 0x7f07d010afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010d4c0 con 0x7f07d0100740 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010d340 con 0x7f07d0101930 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d7244640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010d640 con 0x7f07d01022f0 2024-03-18T23:07:08.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40596/0 (socket says 172.21.15.132:40596) 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 -- 172.21.15.132:0/1523090062 learned_addr learned my addr 172.21.15.132:0/1523090062 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d57e1640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d01022f0 0x7f07d010afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07cffff640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d0101930 0x7f07d010aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 -- 172.21.15.132:0/1523090062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d01022f0 msgr2=0x7f07d010afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d01022f0 0x7f07d010afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.054+0000 7f07d4fe0640 1 -- 172.21.15.132:0/1523090062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d0101930 msgr2=0x7f07d010aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07d4fe0640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d0101930 0x7f07d010aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07d4fe0640 1 -- 172.21.15.132:0/1523090062 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d010b860 con 0x7f07d0100740 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07d4fe0640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07c400d950 tx=0x7f07c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4017070 con 0x7f07d0100740 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4009c30 con 0x7f07d0100740 2024-03-18T23:07:08.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d01a1c80 con 0x7f07d0100740 2024-03-18T23:07:08.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c400f600 con 0x7f07d0100740 2024-03-18T23:07:08.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.055+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01a21d0 con 0x7f07d0100740 2024-03-18T23:07:08.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.056+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d0107a60 con 0x7f07d0100740 2024-03-18T23:07:08.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.057+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f07c401a070 con 0x7f07d0100740 2024-03-18T23:07:08.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.058+0000 7f07cdffb640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 0x7f07c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:08.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.058+0000 7f07cffff640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 0x7f07c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:08.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.058+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f07c40ce030 con 0x7f07d0100740 2024-03-18T23:07:08.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.058+0000 7f07cffff640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 0x7f07c00a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07bc036040 tx=0x7f07bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:08.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.062+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07c4009de0 con 0x7f07d0100740 2024-03-18T23:07:08.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.273+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07d0062420 con 0x7f07d0100740 2024-03-18T23:07:08.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.273+0000 7f07cdffb640 1 -- 172.21.15.132:0/1523090062 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9674 (secure 0 0 0) 0x7f07c409b570 con 0x7f07d0100740 2024-03-18T23:07:08.274 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:08.274 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T23:07:08.274 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 12 2024-03-18T23:07:08.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 msgr2=0x7f07c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:08.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 0x7f07c00a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07bc036040 tx=0x7f07bc005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.276 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 msgr2=0x7f07d010c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07c400d950 tx=0x7f07c400de10 comp rx=0 tx=0).stop 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 shutdown_connections 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f07c00a2880 0x7f07c00a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07d01022f0 0x7f07d010afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07d0101930 0x7f07d010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 --2- 172.21.15.132:0/1523090062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07d0100740 0x7f07d010c450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.275+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 >> 172.21.15.132:0/1523090062 conn(0x7f07d00fbcd0 msgr2=0x7f07d0104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.276+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 shutdown_connections 2024-03-18T23:07:08.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:08.276+0000 7f07d7244640 1 -- 172.21.15.132:0/1523090062 wait complete. 2024-03-18T23:07:08.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:08 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1523090062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:08.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:08 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1523090062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:08.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:08 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1523090062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:08.848 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:08 smithi144 systemd[1]: Starting Ceph mds.b for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:07:09.109 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:08 smithi144 podman[98108]: 2024-03-18 23:07:08.84707624 +0000 UTC m=+0.102947144 container create 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T23:07:09.110 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:08 smithi144 podman[98108]: 2024-03-18 23:07:08.761910943 +0000 UTC m=+0.017781868 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:07:09.110 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 podman[98108]: 2024-03-18 23:07:09.066482915 +0000 UTC m=+0.322353826 container init 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-18T23:07:09.110 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 podman[98108]: 2024-03-18 23:07:09.069225699 +0000 UTC m=+0.325096601 container start 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:07:09.110 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mds[98128]: -- 172.21.15.144:0/2485337653 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637aca7ba40 con 0x5637aca4ec00 2024-03-18T23:07:09.110 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: starting mds.b at 2024-03-18T23:07:09.467 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 bash[98108]: 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 2024-03-18T23:07:09.467 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:09 smithi144 systemd[1]: Started Ceph mds.b for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:09 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:09 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:09.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:09 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:10.289 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:10.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 -- 172.21.15.132:0/733498215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 msgr2=0x7fc764101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 --2- 172.21.15.132:0/733498215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764101c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc758009900 tx=0x7fc75802bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:10.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 -- 172.21.15.132:0/733498215 shutdown_connections 2024-03-18T23:07:10.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 --2- 172.21.15.132:0/733498215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc764109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 --2- 172.21.15.132:0/733498215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 0x7fc764102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 --2- 172.21.15.132:0/733498215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764101c80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 -- 172.21.15.132:0/733498215 >> 172.21.15.132:0/733498215 conn(0x7fc7640fcdd0 msgr2=0x7fc7640ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 -- 172.21.15.132:0/733498215 shutdown_connections 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.393+0000 7fc76ac27640 1 -- 172.21.15.132:0/733498215 wait complete. 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 Processor -- start 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 -- start start 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 0x7fc764198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410d4d0 con 0x7fc764103410 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410d350 con 0x7fc764102a50 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410d650 con 0x7fc764101860 2024-03-18T23:07:10.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40612/0 (socket says 172.21.15.132:40612) 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 -- 172.21.15.132:0/1161627208 learned_addr learned my addr 172.21.15.132:0/1161627208 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc763fff640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 0x7fc764198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7689c3640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 -- 172.21.15.132:0/1161627208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 msgr2=0x7fc764197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 -- 172.21.15.132:0/1161627208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 msgr2=0x7fc764198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 0x7fc764198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 -- 172.21.15.132:0/1161627208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76419f830 con 0x7fc764103410 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7689c3640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc7691c4640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00efd0 tx=0x7fc75c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c003b70 con 0x7fc764103410 2024-03-18T23:07:10.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76419fb20 con 0x7fc764103410 2024-03-18T23:07:10.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c003d10 con 0x7fc764103410 2024-03-18T23:07:10.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.394+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0109f0 con 0x7fc764103410 2024-03-18T23:07:10.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.395+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7641a0070 con 0x7fc764103410 2024-03-18T23:07:10.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.396+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc764108ca0 con 0x7fc764103410 2024-03-18T23:07:10.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.397+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc75c010b90 con 0x7fc764103410 2024-03-18T23:07:10.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.397+0000 7fc761ffb640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 0x7fc7540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.397+0000 7fc7689c3640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 0x7fc7540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.397+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fc75c0cef00 con 0x7fc764103410 2024-03-18T23:07:10.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.398+0000 7fc7689c3640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 0x7fc7540a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7641990f0 tx=0x7fc758005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.401+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc75c09c440 con 0x7fc764103410 2024-03-18T23:07:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:10 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:boot 2024-03-18T23:07:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:10 smithi144 ceph-mon[66781]: fsmap 12 up:standby 2024-03-18T23:07:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:10 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:07:10.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.624+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc76402d1f0 con 0x7fc764103410 2024-03-18T23:07:10.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.625+0000 7fc761ffb640 1 -- 172.21.15.132:0/1161627208 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7fc75c09bd40 con 0x7fc764103410 2024-03-18T23:07:10.625 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:10.626 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-18T23:07:10.626 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 13 2024-03-18T23:07:10.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 msgr2=0x7fc7540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 0x7fc7540a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7641990f0 tx=0x7fc758005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 msgr2=0x7fc76419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00efd0 tx=0x7fc75c00c6a0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 shutdown_connections 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc7540a2800 0x7fc7540a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764103410 0x7fc76419f190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764102a50 0x7fc764198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 --2- 172.21.15.132:0/1161627208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc764101860 0x7fc764197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 >> 172.21.15.132:0/1161627208 conn(0x7fc7640fcdd0 msgr2=0x7fc764105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 shutdown_connections 2024-03-18T23:07:10.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.627+0000 7fc76ac27640 1 -- 172.21.15.132:0/1161627208 wait complete. 2024-03-18T23:07:10.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:10 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:boot 2024-03-18T23:07:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:10 smithi196 ceph-mon[67089]: fsmap 12 up:standby 2024-03-18T23:07:10.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:10 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:07:10.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.747+0000 7fdd4d236640 1 -- 172.21.15.132:0/665668907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48101930 msgr2=0x7fdd48101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.747+0000 7fdd4d236640 1 --2- 172.21.15.132:0/665668907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48101930 0x7fdd48101db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009900 tx=0x7fdd3c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:10.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 -- 172.21.15.132:0/665668907 shutdown_connections 2024-03-18T23:07:10.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 --2- 172.21.15.132:0/665668907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd48108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 --2- 172.21.15.132:0/665668907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48101930 0x7fdd48101db0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 --2- 172.21.15.132:0/665668907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48100740 0x7fdd48100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.752 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 -- 172.21.15.132:0/665668907 >> 172.21.15.132:0/665668907 conn(0x7fdd480fbcb0 msgr2=0x7fdd480fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 -- 172.21.15.132:0/665668907 shutdown_connections 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.748+0000 7fdd4d236640 1 -- 172.21.15.132:0/665668907 wait complete. 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 Processor -- start 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 -- start start 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 0x7fdd48196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd4819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810c3c0 con 0x7fdd48100740 2024-03-18T23:07:10.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810c240 con 0x7fdd48101930 2024-03-18T23:07:10.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4d236640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810c540 con 0x7fdd481022f0 2024-03-18T23:07:10.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd46d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 0x7fdd48196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd4759e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd4819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.749+0000 7fdd46d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40638/0 (socket says 172.21.15.132:40638) 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 -- 172.21.15.132:0/2039533956 learned_addr learned my addr 172.21.15.132:0/2039533956 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 -- 172.21.15.132:0/2039533956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 msgr2=0x7fdd4819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd4819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 -- 172.21.15.132:0/2039533956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 msgr2=0x7fdd48196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 0x7fdd48196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 -- 172.21.15.132:0/2039533956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4819e680 con 0x7fdd48100740 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd4759e640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd4819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd4659c640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 0x7fdd48196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd46d9d640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9c0 tx=0x7fdd3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800cd80 con 0x7fdd48100740 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4819e970 con 0x7fdd48100740 2024-03-18T23:07:10.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4819eee0 con 0x7fdd48100740 2024-03-18T23:07:10.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd380035f0 con 0x7fdd48100740 2024-03-18T23:07:10.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.750+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38010690 con 0x7fdd48100740 2024-03-18T23:07:10.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.751+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd34005290 con 0x7fdd48100740 2024-03-18T23:07:10.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.753+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fdd380037a0 con 0x7fdd48100740 2024-03-18T23:07:10.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.753+0000 7fdd2ffff640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 0x7fdd300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:10.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.754+0000 7fdd4659c640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 0x7fdd300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:10.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.754+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdd380ce970 con 0x7fdd48100740 2024-03-18T23:07:10.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.754+0000 7fdd4659c640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 0x7fdd300a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02c7b0 tx=0x7fdd3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:10.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.758+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdd3809beb0 con 0x7fdd48100740 2024-03-18T23:07:10.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:10 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:boot 2024-03-18T23:07:10.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:10 smithi132 ceph-mon[32020]: fsmap 12 up:standby 2024-03-18T23:07:10.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:10 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:07:10.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.988+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd34005850 con 0x7fdd48100740 2024-03-18T23:07:10.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.989+0000 7fdd2ffff640 1 -- 172.21.15.132:0/2039533956 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18503 (secure 0 0 0) 0x7fdd3809b7b0 con 0x7fdd48100740 2024-03-18T23:07:10.990 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:10.990 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":84,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:06:27.206168+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:10.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 msgr2=0x7fdd300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 0x7fdd300a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02c7b0 tx=0x7fdd3c005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 msgr2=0x7fdd48196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9c0 tx=0x7fdd3800ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 shutdown_connections 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fdd300a2800 0x7fdd300a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd481022f0 0x7fdd4819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd48101930 0x7fdd48196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 --2- 172.21.15.132:0/2039533956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd48100740 0x7fdd48196a60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 >> 172.21.15.132:0/2039533956 conn(0x7fdd480fbcb0 msgr2=0x7fdd48104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 shutdown_connections 2024-03-18T23:07:10.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:10.991+0000 7fdd4d236640 1 -- 172.21.15.132:0/2039533956 wait complete. 2024-03-18T23:07:11.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:07:11.004 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 -- 172.21.15.132:0/3350790527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec1022e0 msgr2=0x7ff1ec108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/3350790527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec108b70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009ca0 tx=0x7ff1e402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 -- 172.21.15.132:0/3350790527 shutdown_connections 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/3350790527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec108b70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/3350790527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec101920 0x7ff1ec101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/3350790527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 0x7ff1ec100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 -- 172.21.15.132:0/3350790527 >> 172.21.15.132:0/3350790527 conn(0x7ff1ec0fbcc0 msgr2=0x7ff1ec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 -- 172.21.15.132:0/3350790527 shutdown_connections 2024-03-18T23:07:11.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.116+0000 7ff1f26a1640 1 -- 172.21.15.132:0/3350790527 wait complete. 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 Processor -- start 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 -- start start 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 0x7ff1ec197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10d510 con 0x7ff1ec101920 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10d390 con 0x7ff1ec1022e0 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1f26a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10d690 con 0x7ff1ec100730 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1eb7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40654/0 (socket says 172.21.15.132:40654) 2024-03-18T23:07:11.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1eb7fe640 1 -- 172.21.15.132:0/1588018846 learned_addr learned my addr 172.21.15.132:0/1588018846 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.117+0000 7ff1ebfff640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 0x7ff1ec197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1f0c3e640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 -- 172.21.15.132:0/1588018846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 msgr2=0x7ff1ec197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 0x7ff1ec197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 -- 172.21.15.132:0/1588018846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec1022e0 msgr2=0x7ff1ec19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 -- 172.21.15.132:0/1588018846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec19f7e0 con 0x7ff1ec101920 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1eb7fe640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000b4a0 tx=0x7ff1e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0014070 con 0x7ff1ec101920 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec19fad0 con 0x7ff1ec101920 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e0010af0 con 0x7ff1ec101920 2024-03-18T23:07:11.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0017d10 con 0x7ff1ec101920 2024-03-18T23:07:11.122 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.118+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1a0020 con 0x7ff1ec101920 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.119+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec107a50 con 0x7ff1ec101920 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.120+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x7ff1ec101920 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.120+0000 7ff1e97fa640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 0x7ff1d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.121+0000 7ff1ebfff640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 0x7ff1d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.121+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff1e00cdd90 con 0x7ff1ec101920 2024-03-18T23:07:11.123 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.121+0000 7ff1ebfff640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 0x7ff1d40a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec101780 tx=0x7ff1dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.124+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1e009b2b0 con 0x7ff1ec101920 2024-03-18T23:07:11.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.314+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1ec02d380 con 0x7ff1ec101920 2024-03-18T23:07:11.315 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.314+0000 7ff1e97fa640 1 -- 172.21.15.132:0/1588018846 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7ff1e009abb0 con 0x7ff1ec101920 2024-03-18T23:07:11.315 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:07:11.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.316+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 msgr2=0x7ff1d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.316+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 0x7ff1d40a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec101780 tx=0x7ff1dc00b390 comp rx=0 tx=0).stop 2024-03-18T23:07:11.317 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 msgr2=0x7ff1ec198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000b4a0 tx=0x7ff1e000b960 comp rx=0 tx=0).stop 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 shutdown_connections 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff1d40a2800 0x7ff1d40a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1ec1022e0 0x7ff1ec19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1ec101920 0x7ff1ec198100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 --2- 172.21.15.132:0/1588018846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1ec100730 0x7ff1ec197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 >> 172.21.15.132:0/1588018846 conn(0x7ff1ec0fbcc0 msgr2=0x7ff1ec104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 shutdown_connections 2024-03-18T23:07:11.318 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.317+0000 7ff1f26a1640 1 -- 172.21.15.132:0/1588018846 wait complete. 2024-03-18T23:07:11.329 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T23:07:11.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 -- 172.21.15.132:0/4130618104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 msgr2=0x7f6218100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 --2- 172.21.15.132:0/4130618104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f6218100b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6214009900 tx=0x7f621402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:11.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 -- 172.21.15.132:0/4130618104 shutdown_connections 2024-03-18T23:07:11.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 --2- 172.21.15.132:0/4130618104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 0x7f6218108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 --2- 172.21.15.132:0/4130618104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 0x7f6218101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 --2- 172.21.15.132:0/4130618104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f6218100b60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 -- 172.21.15.132:0/4130618104 >> 172.21.15.132:0/4130618104 conn(0x7f62180fbcb0 msgr2=0x7f62180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 -- 172.21.15.132:0/4130618104 shutdown_connections 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.433+0000 7f622075b640 1 -- 172.21.15.132:0/4130618104 wait complete. 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 Processor -- start 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 -- start start 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 0x7f621819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 0x7f621819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f621810c3b0 con 0x7f6218100740 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f621810c230 con 0x7f62181022f0 2024-03-18T23:07:11.437 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f622075b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f621810c530 con 0x7f6218101930 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40668/0 (socket says 172.21.15.132:40668) 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 -- 172.21.15.132:0/676749365 learned_addr learned my addr 172.21.15.132:0/676749365 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621dcf6640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 0x7f621819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621ecf8640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 0x7f621819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 -- 172.21.15.132:0/676749365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 msgr2=0x7f621819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 0x7f621819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 -- 172.21.15.132:0/676749365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 msgr2=0x7f621819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 0x7f621819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.434+0000 7f621e4f7640 1 -- 172.21.15.132:0/676749365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621819f830 con 0x7f6218100740 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f621e4f7640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6214000c00 tx=0x7f6214005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214041070 con 0x7f6218100740 2024-03-18T23:07:11.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819fa60 con 0x7f6218100740 2024-03-18T23:07:11.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62140044d0 con 0x7f6218100740 2024-03-18T23:07:11.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214034440 con 0x7f6218100740 2024-03-18T23:07:11.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.435+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621819ffa0 con 0x7f6218100740 2024-03-18T23:07:11.439 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.436+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218068610 con 0x7f6218100740 2024-03-18T23:07:11.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.437+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f6214044070 con 0x7f6218100740 2024-03-18T23:07:11.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.437+0000 7f620f7fe640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 0x7f62040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.437+0000 7f621dcf6640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 0x7f62040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.437+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f62140f00b0 con 0x7f6218100740 2024-03-18T23:07:11.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.438+0000 7f621dcf6640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 0x7f62040a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6218069260 tx=0x7f620800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.442+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62140bcfa0 con 0x7f6218100740 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1161627208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2039533956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:11.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:11 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1588018846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:11.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.627+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f621802d1f0 con 0x7f6218100740 2024-03-18T23:07:11.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.628+0000 7f620f7fe640 1 -- 172.21.15.132:0/676749365 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f62140c1040 con 0x7f6218100740 2024-03-18T23:07:11.628 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:11.628 INFO:teuthology.orchestra.run.smithi132.stdout:[ 2024-03-18T23:07:11.628 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:07:11.629 INFO:teuthology.orchestra.run.smithi132.stdout: "poolnum": 1, 2024-03-18T23:07:11.629 INFO:teuthology.orchestra.run.smithi132.stdout: "poolname": ".mgr" 2024-03-18T23:07:11.629 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:07:11.629 INFO:teuthology.orchestra.run.smithi132.stdout:] 2024-03-18T23:07:11.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 msgr2=0x7f62040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 0x7f62040a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6218069260 tx=0x7f620800b390 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 msgr2=0x7f62180694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6214000c00 tx=0x7f6214005d50 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 shutdown_connections 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f62040a2800 0x7f62040a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62181022f0 0x7f621819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6218101930 0x7f621819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 --2- 172.21.15.132:0/676749365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6218100740 0x7f62180694f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 >> 172.21.15.132:0/676749365 conn(0x7f62180fbcb0 msgr2=0x7f6218104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 shutdown_connections 2024-03-18T23:07:11.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.630+0000 7f622075b640 1 -- 172.21.15.132:0/676749365 wait complete. 2024-03-18T23:07:11.642 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T23:07:11.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T23:07:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1161627208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2039533956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:11 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1588018846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 -- 172.21.15.132:0/3291763771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c102e40 msgr2=0x7f0e0c103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.756 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/3291763771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c102e40 0x7f0e0c103260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009900 tx=0x7f0e0002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 -- 172.21.15.132:0/3291763771 shutdown_connections 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/3291763771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c100270 0x7f0e0c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/3291763771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c0ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/3291763771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c102e40 0x7f0e0c103260 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 -- 172.21.15.132:0/3291763771 >> 172.21.15.132:0/3291763771 conn(0x7f0e0c0fb430 msgr2=0x7f0e0c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 -- 172.21.15.132:0/3291763771 shutdown_connections 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.754+0000 7f0e12ff3640 1 -- 172.21.15.132:0/3291763771 wait complete. 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 Processor -- start 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 -- start start 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 0x7f0e0c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 0x7f0e0c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10d4c0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10d340 con 0x7f0e0c102e40 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c10d640 con 0x7f0e0c100270 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49258/0 (socket says 172.21.15.132:49258) 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 -- 172.21.15.132:0/949859387 learned_addr learned my addr 172.21.15.132:0/949859387 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e0bfff640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 0x7f0e0c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e11590640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 0x7f0e0c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 -- 172.21.15.132:0/949859387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 msgr2=0x7f0e0c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 0x7f0e0c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 -- 172.21.15.132:0/949859387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 msgr2=0x7f0e0c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 0x7f0e0c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 -- 172.21.15.132:0/949859387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c19f7e0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e0bfff640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 0x7f0e0c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:11.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e11590640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 0x7f0e0c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e10d8f640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e00036040 tx=0x7f0e00004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00043020 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0002c980 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.755+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1009c0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.756+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0003e6b0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.756+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c100f10 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.757+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c108d50 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.758+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0e00046020 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.759+0000 7f0e09ffb640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 0x7f0dfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.759+0000 7f0e0bfff640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 0x7f0dfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.759+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0e000f0c40 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.760+0000 7f0e0bfff640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 0x7f0dfc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c1990d0 tx=0x7f0df8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:11.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.764+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0e0004d020 con 0x7f0e0c0ff8b0 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1161627208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2039533956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:07:11.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:11 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1588018846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:07:11.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:11.966+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0e0c02d1f0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:12.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.482+0000 7f0e09ffb640 1 -- 172.21.15.132:0/949859387 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f0e000be2a0 con 0x7f0e0c0ff8b0 2024-03-18T23:07:12.483 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_metadata' created 2024-03-18T23:07:12.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 msgr2=0x7f0dfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:12.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 0x7f0dfc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c1990d0 tx=0x7f0df8005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:12.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 msgr2=0x7f0e0c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e00036040 tx=0x7f0e00004ee0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 shutdown_connections 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0dfc0a2800 0x7f0dfc0a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e0c102e40 0x7f0e0c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0c100270 0x7f0e0c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 --2- 172.21.15.132:0/949859387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e0c0ff8b0 0x7f0e0c197bc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 >> 172.21.15.132:0/949859387 conn(0x7f0e0c0fb430 msgr2=0x7f0e0c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 shutdown_connections 2024-03-18T23:07:12.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.485+0000 7f0e12ff3640 1 -- 172.21.15.132:0/949859387 wait complete. 2024-03-18T23:07:12.500 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T23:07:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:12 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/676749365' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:07:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:12 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 -- 172.21.15.132:0/2644557388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c100740 msgr2=0x7f407c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 --2- 172.21.15.132:0/2644557388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c100740 0x7f407c100b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4070009900 tx=0x7f407002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 -- 172.21.15.132:0/2644557388 shutdown_connections 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 --2- 172.21.15.132:0/2644557388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c1022f0 0x7f407c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 --2- 172.21.15.132:0/2644557388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 --2- 172.21.15.132:0/2644557388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c100740 0x7f407c100b60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 -- 172.21.15.132:0/2644557388 >> 172.21.15.132:0/2644557388 conn(0x7f407c0fbcd0 msgr2=0x7f407c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 -- 172.21.15.132:0/2644557388 shutdown_connections 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.621+0000 7f408307e640 1 -- 172.21.15.132:0/2644557388 wait complete. 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 Processor -- start 2024-03-18T23:07:12.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 -- start start 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 0x7f407c06a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10c390 con 0x7f407c1022f0 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10c210 con 0x7f407c100740 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408307e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f407c10c510 con 0x7f407c101930 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408161b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:12.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f4080e1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 0x7f407c06a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408161b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49282/0 (socket says 172.21.15.132:49282) 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408161b640 1 -- 172.21.15.132:0/2543480499 learned_addr learned my addr 172.21.15.132:0/2543480499 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:37626/0 (socket says 172.21.15.132:37626) 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 -- 172.21.15.132:0/2543480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 msgr2=0x7f407c06a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 0x7f407c06a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 -- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 msgr2=0x7f407c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 -- 172.21.15.132:0/2543480499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c19f770 con 0x7f407c101930 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f408161b640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f4080e1a640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 0x7f407c06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:12.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.622+0000 7f407bfff640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f406800e9f0 tx=0x7f406800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.623+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406800ccc0 con 0x7f407c101930 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.623+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c19fa60 con 0x7f407c101930 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.623+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4068010910 con 0x7f407c101930 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.623+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068018960 con 0x7f407c101930 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.623+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c1a0000 con 0x7f407c101930 2024-03-18T23:07:12.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.624+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c107ae0 con 0x7f407c101930 2024-03-18T23:07:12.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.625+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f4068010ac0 con 0x7f407c101930 2024-03-18T23:07:12.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.626+0000 7f4079ffb640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 0x7f406c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:12.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.626+0000 7f4080e1a640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 0x7f406c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:12.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.626+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f40680ceaa0 con 0x7f407c101930 2024-03-18T23:07:12.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.626+0000 7f4080e1a640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 0x7f406c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4070000c00 tx=0x7f4070005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:12.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.629+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f406809b820 con 0x7f407c101930 2024-03-18T23:07:12.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:12.837+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f407c062420 con 0x7f407c101930 2024-03-18T23:07:12.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:12 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/676749365' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:07:12.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:12 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:07:12.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:12 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/676749365' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T23:07:12.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:12 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T23:07:13.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.494+0000 7f4079ffb640 1 -- 172.21.15.132:0/2543480499 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f406809f8c0 con 0x7f407c101930 2024-03-18T23:07:13.497 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_data' created 2024-03-18T23:07:13.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 msgr2=0x7f406c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 0x7f406c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4070000c00 tx=0x7f4070005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 msgr2=0x7f407c19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f406800e9f0 tx=0x7f406800eeb0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 shutdown_connections 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f406c0a2800 0x7f406c0a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c1022f0 0x7f407c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f407c101930 0x7f407c19c870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 --2- 172.21.15.132:0/2543480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f407c100740 0x7f407c06a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 >> 172.21.15.132:0/2543480499 conn(0x7f407c0fbcd0 msgr2=0x7f407c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 shutdown_connections 2024-03-18T23:07:13.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.498+0000 7f408307e640 1 -- 172.21.15.132:0/2543480499 wait complete. 2024-03-18T23:07:13.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- 172.21.15.132:0/345608264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088101830 msgr2=0x7ff088101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/345608264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088101830 0x7ff088101c50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- 172.21.15.132:0/345608264 shutdown_connections 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/345608264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0881033e0 0x7ff088109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/345608264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/345608264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088101830 0x7ff088101c50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- 172.21.15.132:0/345608264 >> 172.21.15.132:0/345608264 conn(0x7ff0880fcdc0 msgr2=0x7ff0880ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- 172.21.15.132:0/345608264 shutdown_connections 2024-03-18T23:07:13.623 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- 172.21.15.132:0/345608264 wait complete. 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 Processor -- start 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.620+0000 7ff08cc0b640 1 -- start start 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff088101830 0x7ff088197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810d480 con 0x7ff0881033e0 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810d300 con 0x7ff088102a20 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff08810d600 con 0x7ff088101830 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49306/0 (socket says 172.21.15.132:49306) 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 -- 172.21.15.132:0/1970060228 learned_addr learned my addr 172.21.15.132:0/1970060228 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08659c640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff088101830 0x7ff088197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff085d9b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 -- 172.21.15.132:0/1970060228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff088101830 msgr2=0x7ff088197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff088101830 0x7ff088197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 -- 172.21.15.132:0/1970060228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 msgr2=0x7ff088197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 -- 172.21.15.132:0/1970060228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08819f6c0 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff085d9b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff086d9d640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff08000cce0 tx=0x7ff08000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08000f450 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08000fad0 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08819f9b0 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080018c10 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.621+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08819feb0 con 0x7ff0881033e0 2024-03-18T23:07:13.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.622+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff088108b50 con 0x7ff0881033e0 2024-03-18T23:07:13.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.623+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7ff08000f5f0 con 0x7ff0881033e0 2024-03-18T23:07:13.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.624+0000 7ff0777fe640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 0x7ff0700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:13.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.624+0000 7ff08659c640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 0x7ff0700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:13.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.624+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7ff0800ceff0 con 0x7ff0881033e0 2024-03-18T23:07:13.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.624+0000 7ff08659c640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 0x7ff0700a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:13.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.628+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff080014030 con 0x7ff0881033e0 2024-03-18T23:07:13.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:13.851+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff08802d1f0 con 0x7ff0881033e0 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2543480499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:13.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:13 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:07:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:07:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2543480499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:13.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:13 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/949859387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2543480499' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:13.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:07:13.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:07:13.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:13 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:14.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.834+0000 7ff0777fe640 1 -- 172.21.15.132:0/1970060228 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff08009c2d0 con 0x7ff0881033e0 2024-03-18T23:07:14.835 INFO:teuthology.orchestra.run.smithi132.stderr:new fs with metadata pool 2 and data pool 3 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 msgr2=0x7ff0700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 0x7ff0700a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff07c000c00 tx=0x7ff07c004f10 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 msgr2=0x7ff08819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff08000cce0 tx=0x7ff08000a620 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 shutdown_connections 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff0700a2800 0x7ff0700a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0881033e0 0x7ff08819f050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff088102a20 0x7ff088197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 --2- 172.21.15.132:0/1970060228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff088101830 0x7ff088197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 >> 172.21.15.132:0/1970060228 conn(0x7ff0880fcdc0 msgr2=0x7ff088105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:14.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 shutdown_connections 2024-03-18T23:07:14.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.837+0000 7ff08cc0b640 1 -- 172.21.15.132:0/1970060228 wait complete. 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:14.853 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:14 smithi132 ceph-mon[32020]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:14.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:14.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:14 smithi144 ceph-mon[66781]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:14.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 -- 172.21.15.132:0/1584940609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768101950 msgr2=0x7f8768101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1584940609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768101950 0x7f8768101dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f875c009900 tx=0x7f875c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 -- 172.21.15.132:0/1584940609 shutdown_connections 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1584940609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768102310 0x7f8768108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1584940609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768101950 0x7f8768101dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1584940609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8768100760 0x7f8768100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 -- 172.21.15.132:0/1584940609 >> 172.21.15.132:0/1584940609 conn(0x7f87680fbcd0 msgr2=0x7f87680fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 -- 172.21.15.132:0/1584940609 shutdown_connections 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.960+0000 7f876d5f1640 1 -- 172.21.15.132:0/1584940609 wait complete. 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 Processor -- start 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 -- start start 2024-03-18T23:07:14.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768100760 0x7f876806a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f876819d0b0 0x7f876819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f876810c3e0 con 0x7f8768102310 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f876810c260 con 0x7f8768100760 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f876d5f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f876810c560 con 0x7f876819d0b0 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49322/0 (socket says 172.21.15.132:49322) 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 -- 172.21.15.132:0/1600757010 learned_addr learned my addr 172.21.15.132:0/1600757010 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f8766ffd640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768100760 0x7f876806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 -- 172.21.15.132:0/1600757010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f876819d0b0 msgr2=0x7f876819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f876819d0b0 0x7f876819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 -- 172.21.15.132:0/1600757010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768100760 msgr2=0x7f876806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:14.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768100760 0x7f876806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 -- 172.21.15.132:0/1600757010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876819fe00 con 0x7f8768102310 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.961+0000 7f87667fc640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f875c036040 tx=0x7f875c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.962+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c041070 con 0x7f8768102310 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.962+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87681a0090 con 0x7f8768102310 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.962+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875c0044d0 con 0x7f8768102310 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.962+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c034440 con 0x7f8768102310 2024-03-18T23:07:14.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.962+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681a0580 con 0x7f8768102310 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.963+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8768107b40 con 0x7f8768102310 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.964+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f875c044070 con 0x7f8768102310 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.964+0000 7f874ffff640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 0x7f87500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.964+0000 7f8766ffd640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 0x7f87500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.964+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f875c0f0400 con 0x7f8768102310 2024-03-18T23:07:14.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.965+0000 7f8766ffd640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 0x7f87500a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f875800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:14.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:14.969+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f875c04b030 con 0x7f8768102310 2024-03-18T23:07:14.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:07:14.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:14 smithi196 ceph-mon[67089]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T23:07:15.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.150+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f876802d3a0 con 0x7f8768102310 2024-03-18T23:07:15.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.152+0000 7f874ffff640 1 -- 172.21.15.132:0/1600757010 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22266 (secure 0 0 0) 0x7f875c0bd5f0 con 0x7f8768102310 2024-03-18T23:07:15.152 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:15.153 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:07:15.045318+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:15.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 msgr2=0x7f87500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 0x7f87500a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f875800b390 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 msgr2=0x7f876819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f875c036040 tx=0x7f875c02c640 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 shutdown_connections 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f87500a2800 0x7f87500a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f876819d0b0 0x7f876819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8768102310 0x7f876819cb70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 --2- 172.21.15.132:0/1600757010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8768100760 0x7f876806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 >> 172.21.15.132:0/1600757010 conn(0x7f87680fbcd0 msgr2=0x7f8768104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 shutdown_connections 2024-03-18T23:07:15.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.154+0000 7f876d5f1640 1 -- 172.21.15.132:0/1600757010 wait complete. 2024-03-18T23:07:15.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:15.269 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1839046178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c101930 msgr2=0x7f1b9c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1839046178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c101930 0x7f1b9c101db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009900 tx=0x7f1b8c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1839046178 shutdown_connections 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1839046178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1839046178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c101930 0x7f1b9c101db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1839046178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c100740 0x7f1b9c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.266+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1839046178 >> 172.21.15.132:0/1839046178 conn(0x7f1b9c0fbcb0 msgr2=0x7f1b9c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1839046178 shutdown_connections 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1839046178 wait complete. 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 Processor -- start 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- start start 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 0x7f1b9c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.270 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 0x7f1b9c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c10d5c0 con 0x7f1b9c1022f0 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c10d440 con 0x7f1b9c101930 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba44a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c10d740 con 0x7f1b9c100740 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba2a3e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba2a3e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49340/0 (socket says 172.21.15.132:49340) 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba2a3e640 1 -- 172.21.15.132:0/1175910256 learned_addr learned my addr 172.21.15.132:0/1175910256 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba1a3c640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 0x7f1b9c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba2a3e640 1 -- 172.21.15.132:0/1175910256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 msgr2=0x7f1b9c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.267+0000 7f1ba223d640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 0x7f1b9c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba2a3e640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 0x7f1b9c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba2a3e640 1 -- 172.21.15.132:0/1175910256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 msgr2=0x7f1b9c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.271 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba2a3e640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 0x7f1b9c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba2a3e640 1 -- 172.21.15.132:0/1175910256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c19f890 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba223d640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 0x7f1b9c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba1a3c640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 0x7f1b9c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba2a3e640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400b970 tx=0x7f1b9400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94019070 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c19fb80 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9400cb40 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94007610 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.268+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1a00d0 con 0x7f1b9c1022f0 2024-03-18T23:07:15.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.269+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c107a60 con 0x7f1b9c1022f0 2024-03-18T23:07:15.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.270+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f1b9401c070 con 0x7f1b9c1022f0 2024-03-18T23:07:15.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.270+0000 7f1b937fe640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 0x7f1b880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.271+0000 7f1ba223d640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 0x7f1b880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.271+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1b940ce480 con 0x7f1b9c1022f0 2024-03-18T23:07:15.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.271+0000 7f1ba223d640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 0x7f1b880a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.274+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b9409b670 con 0x7f1b9c1022f0 2024-03-18T23:07:15.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.451+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b9c02d1f0 con 0x7f1b9c1022f0 2024-03-18T23:07:15.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.453+0000 7f1b937fe640 1 -- 172.21.15.132:0/1175910256 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22266 (secure 0 0 0) 0x7f1b9409af70 con 0x7f1b9c1022f0 2024-03-18T23:07:15.453 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:15.454 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:07:15.045318+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:15.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 msgr2=0x7f1b880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 0x7f1b880a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 msgr2=0x7f1b9c19f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400b970 tx=0x7f1b9400be30 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 shutdown_connections 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b880a2800 0x7f1b880a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b9c1022f0 0x7f1b9c19f1f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9c101930 0x7f1b9c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 --2- 172.21.15.132:0/1175910256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b9c100740 0x7f1b9c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 >> 172.21.15.132:0/1175910256 conn(0x7f1b9c0fbcb0 msgr2=0x7f1b9c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 shutdown_connections 2024-03-18T23:07:15.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.455+0000 7f1ba44a1640 1 -- 172.21.15.132:0/1175910256 wait complete. 2024-03-18T23:07:15.467 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T23:07:15.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- 172.21.15.132:0/2651584969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c101930 msgr2=0x7f888c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 --2- 172.21.15.132:0/2651584969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c101930 0x7f888c101db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f887c009900 tx=0x7f887c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- 172.21.15.132:0/2651584969 shutdown_connections 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 --2- 172.21.15.132:0/2651584969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c1022f0 0x7f888c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 --2- 172.21.15.132:0/2651584969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c101930 0x7f888c101db0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 --2- 172.21.15.132:0/2651584969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c100740 0x7f888c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- 172.21.15.132:0/2651584969 >> 172.21.15.132:0/2651584969 conn(0x7f888c0fbcd0 msgr2=0x7f888c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- 172.21.15.132:0/2651584969 shutdown_connections 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- 172.21.15.132:0/2651584969 wait complete. 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 Processor -- start 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.572+0000 7f88944c6640 1 -- start start 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c1022f0 0x7f888c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10d4e0 con 0x7f888c100740 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10d360 con 0x7f888c101930 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10d660 con 0x7f888c1022f0 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49368/0 (socket says 172.21.15.132:49368) 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 -- 172.21.15.132:0/1206748864 learned_addr learned my addr 172.21.15.132:0/1206748864 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8891a61640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892a63640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c1022f0 0x7f888c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 -- 172.21.15.132:0/1206748864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c1022f0 msgr2=0x7f888c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c1022f0 0x7f888c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 -- 172.21.15.132:0/1206748864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 msgr2=0x7f888c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:15.576 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 -- 172.21.15.132:0/1206748864 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c19f780 con 0x7f888c100740 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f8892262640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f888800c970 tx=0x7f888800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888004220 con 0x7f888c100740 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88880043c0 con 0x7f888c100740 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c19fa70 con 0x7f888c100740 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888004a00 con 0x7f888c100740 2024-03-18T23:07:15.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.573+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c19ff40 con 0x7f888c100740 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.574+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888c107a60 con 0x7f888c100740 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.575+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f88880079b0 con 0x7f888c100740 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.576+0000 7f88837fe640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 0x7f88780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.576+0000 7f8891a61640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 0x7f88780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.576+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f88880ce5e0 con 0x7f888c100740 2024-03-18T23:07:15.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.576+0000 7f8891a61640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 0x7f88780a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f887c005ab0 tx=0x7f887c004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:15.581 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.581+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8888014030 con 0x7f888c100740 2024-03-18T23:07:15.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.792+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f888c062420 con 0x7f888c100740 2024-03-18T23:07:16.481 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: pgmap v285: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1600757010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1175910256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: pgmap v285: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:16.482 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:16.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1600757010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1175910256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.483 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.840+0000 7f88837fe640 1 -- 172.21.15.132:0/1206748864 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f888809b180 con 0x7f888c100740 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 msgr2=0x7f88780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 0x7f88780a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f887c005ab0 tx=0x7f887c004f10 comp rx=0 tx=0).stop 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 msgr2=0x7f888c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f888800c970 tx=0x7f888800ce30 comp rx=0 tx=0).stop 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 shutdown_connections 2024-03-18T23:07:16.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f88780a2800 0x7f88780a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f888c1022f0 0x7f888c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 --2- 172.21.15.132:0/1206748864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f888c100740 0x7f888c197b60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 >> 172.21.15.132:0/1206748864 conn(0x7f888c0fbcd0 msgr2=0x7f888c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 shutdown_connections 2024-03-18T23:07:16.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:15.843+0000 7f88944c6640 1 -- 172.21.15.132:0/1206748864 wait complete. 2024-03-18T23:07:16.485 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-18T23:07:16.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1970060228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: pgmap v285: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1600757010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1175910256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:16.486 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 -- 172.21.15.132:0/1552938791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54101cd0 msgr2=0x7fce541020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 --2- 172.21.15.132:0/1552938791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54101cd0 0x7fce541020f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce44009900 tx=0x7fce4402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 -- 172.21.15.132:0/1552938791 shutdown_connections 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 --2- 172.21.15.132:0/1552938791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce541013c0 0x7fce54108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 --2- 172.21.15.132:0/1552938791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54100a00 0x7fce54100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 --2- 172.21.15.132:0/1552938791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54101cd0 0x7fce541020f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 -- 172.21.15.132:0/1552938791 >> 172.21.15.132:0/1552938791 conn(0x7fce540fb450 msgr2=0x7fce540fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:16.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 -- 172.21.15.132:0/1552938791 shutdown_connections 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.600+0000 7fce59fc6640 1 -- 172.21.15.132:0/1552938791 wait complete. 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 Processor -- start 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 -- start start 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 0x7fce54100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54101cd0 0x7fce5419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fce5410c410 con 0x7fce541013c0 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce5410c290 con 0x7fce54101cd0 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce59fc6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce5410c590 con 0x7fce54100a00 2024-03-18T23:07:16.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce52ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce52ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49388/0 (socket says 172.21.15.132:49388) 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce52ffd640 1 -- 172.21.15.132:0/2600735460 learned_addr learned my addr 172.21.15.132:0/2600735460 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.601+0000 7fce53fff640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54101cd0 0x7fce5419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce537fe640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 0x7fce54100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 -- 172.21.15.132:0/2600735460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 msgr2=0x7fce54100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 0x7fce54100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 -- 172.21.15.132:0/2600735460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54101cd0 msgr2=0x7fce5419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54101cd0 0x7fce5419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 -- 172.21.15.132:0/2600735460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5419f810 con 0x7fce541013c0 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce537fe640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 0x7fce54100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:07:16.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce52ffd640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ea60 tx=0x7fce4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800cde0 con 0x7fce541013c0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce480035f0 con 0x7fce541013c0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5419faa0 con 0x7fce541013c0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48010670 con 0x7fce541013c0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.602+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5419fff0 con 0x7fce541013c0 2024-03-18T23:07:16.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.603+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce54107b40 con 0x7fce541013c0 2024-03-18T23:07:16.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.604+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fce480037a0 con 0x7fce541013c0 2024-03-18T23:07:16.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.604+0000 7fce50ff9640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 0x7fce3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.605+0000 7fce537fe640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 0x7fce3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.605+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fce480cecb0 con 0x7fce541013c0 2024-03-18T23:07:16.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.605+0000 7fce537fe640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 0x7fce3c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce440096c0 tx=0x7fce4402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:16.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.609+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce4809bea0 con 0x7fce541013c0 2024-03-18T23:07:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.820+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fce5402d400 con 0x7fce541013c0 2024-03-18T23:07:16.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.847+0000 7fce50ff9640 1 -- 172.21.15.132:0/2600735460 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7fce4809b7a0 con 0x7fce541013c0 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 msgr2=0x7fce3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 0x7fce3c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fce440096c0 tx=0x7fce4402df90 comp rx=0 tx=0).stop 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 msgr2=0x7fce5419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ea60 tx=0x7fce4800ef20 comp rx=0 tx=0).stop 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 shutdown_connections 2024-03-18T23:07:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fce3c0a2800 0x7fce3c0a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce54101cd0 0x7fce5419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce541013c0 0x7fce5419c8e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 --2- 172.21.15.132:0/2600735460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce54100a00 0x7fce54100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 >> 172.21.15.132:0/2600735460 conn(0x7fce540fb450 msgr2=0x7fce54104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 shutdown_connections 2024-03-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.849+0000 7fce59fc6640 1 -- 172.21.15.132:0/2600735460 wait complete. 2024-03-18T23:07:16.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-18T23:07:16.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.966+0000 7fb58deac640 1 -- 172.21.15.132:0/795337318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb588101930 msgr2=0x7fb588101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.966+0000 7fb58deac640 1 --2- 172.21.15.132:0/795337318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb588101930 0x7fb588101db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb570009900 tx=0x7fb57002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:16.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 -- 172.21.15.132:0/795337318 shutdown_connections 2024-03-18T23:07:16.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- 172.21.15.132:0/795337318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5881022f0 0x7fb588108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- 172.21.15.132:0/795337318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb588101930 0x7fb588101db0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- 172.21.15.132:0/795337318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588100740 0x7fb588100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 -- 172.21.15.132:0/795337318 >> 172.21.15.132:0/795337318 conn(0x7fb5880fbcd0 msgr2=0x7fb5880fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 -- 172.21.15.132:0/795337318 shutdown_connections 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 -- 172.21.15.132:0/795337318 wait complete. 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 Processor -- start 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 -- start start 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 0x7fb588197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 0x7fb588198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.967+0000 7fb58deac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb58deac640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810c3a0 con 0x7fb5881022f0 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb58deac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810c220 con 0x7fb588100740 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb58deac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810c520 con 0x7fb588101930 2024-03-18T23:07:16.970 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb587fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 0x7fb588197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb587fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49408/0 (socket says 172.21.15.132:49408) 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb587fff640 1 -- 172.21.15.132:0/4005208141 learned_addr learned my addr 172.21.15.132:0/4005208141 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb586ffd640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 0x7fb588198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 -- 172.21.15.132:0/4005208141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 msgr2=0x7fb588198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 0x7fb588198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 -- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 msgr2=0x7fb58819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 -- 172.21.15.132:0/4005208141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58819f6e0 con 0x7fb588100740 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb586ffd640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 0x7fb588198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb587fff640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb5877fe640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 0x7fb588197b10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb58000ccb0 tx=0x7fb58000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:16.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580003970 con 0x7fb588100740 2024-03-18T23:07:16.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58819f9d0 con 0x7fb588100740 2024-03-18T23:07:16.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580004030 con 0x7fb588100740 2024-03-18T23:07:16.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000f830 con 0x7fb588100740 2024-03-18T23:07:16.972 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.968+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58819ff20 con 0x7fb588100740 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.969+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5881a0150 con 0x7fb588100740 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.970+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb5800041e0 con 0x7fb588100740 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.971+0000 7fb584ff9640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 0x7fb5740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.971+0000 7fb586ffd640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 0x7fb5740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.971+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb5800cf0b0 con 0x7fb588100740 2024-03-18T23:07:16.973 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.971+0000 7fb586ffd640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 0x7fb5740a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb570036040 tx=0x7fb570000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:16.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:16.975+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb580014070 con 0x7fb588100740 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: Cluster is now healthy 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:active 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:17.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:16 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-18T23:07:17.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:07:17.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: Cluster is now healthy 2024-03-18T23:07:17.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:07:17.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:active 2024-03-18T23:07:17.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:17.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:16 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: Cluster is now healthy 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1206748864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] up:active 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:17.125 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:16 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-18T23:07:17.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.189+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb5881a0320 con 0x7fb588100740 2024-03-18T23:07:17.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.858+0000 7fb584ff9640 1 -- 172.21.15.132:0/4005208141 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fb58009c2a0 con 0x7fb588100740 2024-03-18T23:07:17.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 msgr2=0x7fb5740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 0x7fb5740a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb570036040 tx=0x7fb570000bc0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 msgr2=0x7fb588197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 0x7fb588197b10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb58000ccb0 tx=0x7fb58000a620 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 shutdown_connections 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb5740a2800 0x7fb5740a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5881022f0 0x7fb58819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588101930 0x7fb588198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 --2- 172.21.15.132:0/4005208141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb588100740 0x7fb588197b10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 >> 172.21.15.132:0/4005208141 conn(0x7fb5880fbcd0 msgr2=0x7fb5880fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 shutdown_connections 2024-03-18T23:07:17.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.861+0000 7fb58deac640 1 -- 172.21.15.132:0/4005208141 wait complete. 2024-03-18T23:07:17.876 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-18T23:07:17.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 -- 172.21.15.132:0/2552506647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf8103490 msgr2=0x7efcf8109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 --2- 172.21.15.132:0/2552506647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf8103490 0x7efcf8109d20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009cd0 tx=0x7efcf002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 -- 172.21.15.132:0/2552506647 shutdown_connections 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 --2- 172.21.15.132:0/2552506647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf8103490 0x7efcf8109d20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 --2- 172.21.15.132:0/2552506647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8102ad0 0x7efcf8102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 --2- 172.21.15.132:0/2552506647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf80ffb00 0x7efcf80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 -- 172.21.15.132:0/2552506647 >> 172.21.15.132:0/2552506647 conn(0x7efcf80fb430 msgr2=0x7efcf80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 -- 172.21.15.132:0/2552506647 shutdown_connections 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.982+0000 7efcffae8640 1 -- 172.21.15.132:0/2552506647 wait complete. 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 Processor -- start 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 -- start start 2024-03-18T23:07:17.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf8102ad0 0x7efcf8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8103490 0x7efcf819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810d4d0 con 0x7efcf80ffb00 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810d350 con 0x7efcf8103490 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcffae8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810d650 con 0x7efcf8102ad0 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49422/0 (socket says 172.21.15.132:49422) 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 -- 172.21.15.132:0/2886508098 learned_addr learned my addr 172.21.15.132:0/2886508098 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd083640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf8102ad0 0x7efcf8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfe085640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8103490 0x7efcf819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 -- 172.21.15.132:0/2886508098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf8102ad0 msgr2=0x7efcf8198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf8102ad0 0x7efcf8198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 -- 172.21.15.132:0/2886508098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8103490 msgr2=0x7efcf819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:17.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8103490 0x7efcf819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 -- 172.21.15.132:0/2886508098 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf819f730 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.983+0000 7efcfd884640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efcf400e9c0 tx=0x7efcf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.984+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf400cd80 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.984+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf40035f0 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.984+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf819fa20 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.984+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4010690 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.984+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf819ff70 con 0x7efcf80ffb00 2024-03-18T23:07:17.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.985+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8108c00 con 0x7efcf80ffb00 2024-03-18T23:07:17.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.986+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7efcf40037a0 con 0x7efcf80ffb00 2024-03-18T23:07:17.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.986+0000 7efceeffd640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 0x7efce40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:17.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.986+0000 7efcfd083640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 0x7efce40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:17.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.986+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7efcf40cebe0 con 0x7efcf80ffb00 2024-03-18T23:07:17.989 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.987+0000 7efcfd083640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 0x7efce40a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efcf8199020 tx=0x7efce8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:17.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:17.990+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efcf409bdd0 con 0x7efcf80ffb00 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:07:18.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: pgmap v287: 97 pgs: 11 creating+peering, 59 active+clean, 27 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 898 B/s wr, 2 op/s 2024-03-18T23:07:18.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4005208141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.110 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:17 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:07:18.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-18T23:07:18.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:07:18.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:18.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:07:18.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:07:18.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: pgmap v287: 97 pgs: 11 creating+peering, 59 active+clean, 27 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 898 B/s wr, 2 op/s 2024-03-18T23:07:18.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4005208141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:17 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.205+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7efcf802d480 con 0x7efcf80ffb00 2024-03-18T23:07:18.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T23:07:18.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2600735460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-18T23:07:18.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T23:07:18.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T23:07:18.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-18T23:07:18.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-18T23:07:18.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: pgmap v287: 97 pgs: 11 creating+peering, 59 active+clean, 27 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 898 B/s wr, 2 op/s 2024-03-18T23:07:18.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4005208141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:17 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T23:07:18.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.738+0000 7efceeffd640 1 -- 172.21.15.132:0/2886508098 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7efcf409b6d0 con 0x7efcf80ffb00 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 msgr2=0x7efce40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 0x7efce40a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efcf8199020 tx=0x7efce8005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 msgr2=0x7efcf8197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efcf400e9c0 tx=0x7efcf400ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 shutdown_connections 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7efce40a2880 0x7efce40a4d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcf8103490 0x7efcf819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efcf8102ad0 0x7efcf8198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 --2- 172.21.15.132:0/2886508098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf80ffb00 0x7efcf8197b10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 >> 172.21.15.132:0/2886508098 conn(0x7efcf80fb430 msgr2=0x7efcf8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 shutdown_connections 2024-03-18T23:07:18.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.740+0000 7efcffae8640 1 -- 172.21.15.132:0/2886508098 wait complete. 2024-03-18T23:07:18.753 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-18T23:07:18.753 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-18T23:07:18.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.857+0000 7f7c48c26640 1 -- 172.21.15.132:0/4193663216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 msgr2=0x7f7c44101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.857+0000 7f7c48c26640 1 --2- 172.21.15.132:0/4193663216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44101d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009900 tx=0x7f7c3802bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:18.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 -- 172.21.15.132:0/4193663216 shutdown_connections 2024-03-18T23:07:18.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 --2- 172.21.15.132:0/4193663216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c441022d0 0x7f7c44108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 --2- 172.21.15.132:0/4193663216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44101d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 --2- 172.21.15.132:0/4193663216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c44100720 0x7f7c44100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 -- 172.21.15.132:0/4193663216 >> 172.21.15.132:0/4193663216 conn(0x7f7c440fbcb0 msgr2=0x7f7c440fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 -- 172.21.15.132:0/4193663216 shutdown_connections 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 -- 172.21.15.132:0/4193663216 wait complete. 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 Processor -- start 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.858+0000 7f7c48c26640 1 -- start start 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 0x7f7c44197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c441022d0 0x7f7c4419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410c3a0 con 0x7f7c44101910 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410c220 con 0x7f7c441022d0 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c48c26640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410c520 con 0x7f7c44100720 2024-03-18T23:07:18.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c41d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c41d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49434/0 (socket says 172.21.15.132:49434) 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c41d9b640 1 -- 172.21.15.132:0/2606834636 learned_addr learned my addr 172.21.15.132:0/2606834636 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 0x7f7c44197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c42d9d640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c441022d0 0x7f7c4419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 -- 172.21.15.132:0/2606834636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c441022d0 msgr2=0x7f7c4419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c441022d0 0x7f7c4419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 -- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 msgr2=0x7f7c44198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 -- 172.21.15.132:0/2606834636 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419f750 con 0x7f7c44100720 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c41d9b640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:18.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.859+0000 7f7c4259c640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 0x7f7c44197b30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9c0 tx=0x7f7c3c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:18.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.862+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00ccc0 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.862+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c0035f0 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.862+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4419fa40 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.862+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00ce60 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.862+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4419ffb0 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.863+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c441a0240 con 0x7f7c44100720 2024-03-18T23:07:18.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.864+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f7c3c0037a0 con 0x7f7c44100720 2024-03-18T23:07:18.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.865+0000 7f7c337fe640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 0x7f7c2c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:18.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.865+0000 7f7c41d9b640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 0x7f7c2c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:18.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.865+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7c3c0cec50 con 0x7f7c44100720 2024-03-18T23:07:18.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.865+0000 7f7c41d9b640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 0x7f7c2c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c3802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:18.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:18.868+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7c3c09be40 con 0x7f7c44100720 2024-03-18T23:07:19.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.069+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7c441a0410 con 0x7f7c2c0a2880 2024-03-18T23:07:19.075 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:07:19.075 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:active 2024-03-18T23:07:19.075 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:07:19.075 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: Cluster is now healthy 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:active 2024-03-18T23:07:19.076 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:18 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:07:19.094 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.094+0000 7f7c337fe640 1 -- 172.21.15.132:0/2606834636 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c441a0410 con 0x7f7c2c0a2880 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 msgr2=0x7f7c2c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 0x7f7c2c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000c00 tx=0x7f7c3802c880 comp rx=0 tx=0).stop 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 msgr2=0x7f7c44197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 0x7f7c44197b30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9c0 tx=0x7f7c3c00ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 shutdown_connections 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7c2c0a2880 0x7f7c2c0a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c441022d0 0x7f7c4419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c44101910 0x7f7c44198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 --2- 172.21.15.132:0/2606834636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c44100720 0x7f7c44197b30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 >> 172.21.15.132:0/2606834636 conn(0x7f7c440fbcb0 msgr2=0x7f7c440fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 shutdown_connections 2024-03-18T23:07:19.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.096+0000 7f7c48c26640 1 -- 172.21.15.132:0/2606834636 wait complete. 2024-03-18T23:07:19.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-18T23:07:19.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- 172.21.15.132:0/4280792732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d81022e0 msgr2=0x7f09d8108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/4280792732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d81022e0 0x7f09d8108b70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009ca0 tx=0x7f09d002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- 172.21.15.132:0/4280792732 shutdown_connections 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/4280792732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d81022e0 0x7f09d8108b70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/4280792732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8101920 0x7f09d8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/4280792732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8100730 0x7f09d8100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- 172.21.15.132:0/4280792732 >> 172.21.15.132:0/4280792732 conn(0x7f09d80fbcc0 msgr2=0x7f09d80fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- 172.21.15.132:0/4280792732 shutdown_connections 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- 172.21.15.132:0/4280792732 wait complete. 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 Processor -- start 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.215+0000 7f09ddfdf640 1 -- start start 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8100730 0x7f09d8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d81022e0 0x7f09d819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f09d810c390 con 0x7f09d8101920 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09d810c210 con 0x7f09d81022e0 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f09d810c510 con 0x7f09d8100730 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49482/0 (socket says 172.21.15.132:49482) 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 -- 172.21.15.132:0/2628045980 learned_addr learned my addr 172.21.15.132:0/2628045980 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09d7fff640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d81022e0 0x7f09d819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09d77fe640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8100730 0x7f09d8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 -- 172.21.15.132:0/2628045980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8100730 msgr2=0x7f09d8196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8100730 0x7f09d8196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 -- 172.21.15.132:0/2628045980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d81022e0 msgr2=0x7f09d819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d81022e0 0x7f09d819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 -- 172.21.15.132:0/2628045980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d819e620 con 0x7f09d8101920 2024-03-18T23:07:19.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ceffd640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09c000eb70 tx=0x7f09c000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c000be60 con 0x7f09d8101920 2024-03-18T23:07:19.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c00035f0 con 0x7f09d8101920 2024-03-18T23:07:19.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00146e0 con 0x7f09d8101920 2024-03-18T23:07:19.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d8069ed0 con 0x7f09d8101920 2024-03-18T23:07:19.220 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.216+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d806a420 con 0x7f09d8101920 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.217+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d8068a60 con 0x7f09d8101920 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.218+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f09c00037a0 con 0x7f09d8101920 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.219+0000 7f09d57fa640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 0x7f09c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.219+0000 7f09d77fe640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 0x7f09c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.219+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f09c00d08b0 con 0x7f09d8101920 2024-03-18T23:07:19.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.219+0000 7f09d77fe640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 0x7f09c40a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f09d8101780 tx=0x7f09c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.223 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.223+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f09c009d430 con 0x7f09d8101920 2024-03-18T23:07:19.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:active 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: Cluster is now healthy 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:active 2024-03-18T23:07:19.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:18 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] up:active 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: Cluster is now healthy 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-18T23:07:19.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-18T23:07:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T23:07:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2886508098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T23:07:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] up:active 2024-03-18T23:07:19.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:18 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:07:19.365 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:07:19.073+0000 7faf06313700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:07:19.365 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:07:19.073+0000 7faf06313700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:07:19.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:07:19.073+0000 7faf06313700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:07:19.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:07:19.073+0000 7faf06313700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:07:19.366 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:07:19.073+0000 7faf06313700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:07:19.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.424+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f09d806a680 con 0x7f09c40a2800 2024-03-18T23:07:19.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.446+0000 7f09d57fa640 1 -- 172.21.15.132:0/2628045980 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09d806a680 con 0x7f09c40a2800 2024-03-18T23:07:19.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 msgr2=0x7f09c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 0x7f09c40a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f09d8101780 tx=0x7f09c8005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:19.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 msgr2=0x7f09d8196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.448 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09c000eb70 tx=0x7f09c000b690 comp rx=0 tx=0).stop 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 shutdown_connections 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f09c40a2800 0x7f09c40a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d81022e0 0x7f09d819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09d8101920 0x7f09d8196f40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.447+0000 7f09ddfdf640 1 --2- 172.21.15.132:0/2628045980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09d8100730 0x7f09d8196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.448+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 >> 172.21.15.132:0/2628045980 conn(0x7f09d80fbcc0 msgr2=0x7f09d8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.448+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 shutdown_connections 2024-03-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.448+0000 7f09ddfdf640 1 -- 172.21.15.132:0/2628045980 wait complete. 2024-03-18T23:07:19.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 -- 172.21.15.132:0/2974712101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4104d10 msgr2=0x7f32a4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 --2- 172.21.15.132:0/2974712101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4104d10 0x7f32a4107100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3294009900 tx=0x7f329402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 -- 172.21.15.132:0/2974712101 shutdown_connections 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 --2- 172.21.15.132:0/2974712101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4107640 0x7f32a4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 --2- 172.21.15.132:0/2974712101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4104d10 0x7f32a4107100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 --2- 172.21.15.132:0/2974712101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 -- 172.21.15.132:0/2974712101 >> 172.21.15.132:0/2974712101 conn(0x7f32a40fbd30 msgr2=0x7f32a40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 -- 172.21.15.132:0/2974712101 shutdown_connections 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.565+0000 7f32ab922640 1 -- 172.21.15.132:0/2974712101 wait complete. 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 Processor -- start 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 -- start start 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f32a410d4c0 con 0x7f32a4107640 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32a410d340 con 0x7f32a4104d10 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32ab922640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f32a410d640 con 0x7f32a41023e0 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a9ebf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a8ebd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a8ebd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:35610/0 (socket says 172.21.15.132:35610) 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a8ebd640 1 -- 172.21.15.132:0/4251882963 learned_addr learned my addr 172.21.15.132:0/4251882963 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a9ebf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49504/0 (socket says 172.21.15.132:49504) 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 -- 172.21.15.132:0/4251882963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 msgr2=0x7f32a4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 -- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 msgr2=0x7f32a419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a96be640 1 -- 172.21.15.132:0/4251882963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a419f810 con 0x7f32a41023e0 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a8ebd640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.566+0000 7f32a9ebf640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f32a96be640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a4197bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009b50 tx=0x7f32a000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009e50 con 0x7f32a41023e0 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a4100230 con 0x7f32a41023e0 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a0005500 con 0x7f32a41023e0 2024-03-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0019670 con 0x7f32a41023e0 2024-03-18T23:07:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.567+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a4100750 con 0x7f32a41023e0 2024-03-18T23:07:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.568+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a40fd820 con 0x7f32a41023e0 2024-03-18T23:07:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.569+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f32a0020020 con 0x7f32a41023e0 2024-03-18T23:07:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.569+0000 7f329a7fc640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 0x7f32900a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.569+0000 7f32a8ebd640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 0x7f32900a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.569+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f32a00d2130 con 0x7f32a41023e0 2024-03-18T23:07:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.570+0000 7f32a8ebd640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 0x7f32900a4e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f329402c8e0 tx=0x7f3294005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.574+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f32a009f320 con 0x7f32a41023e0 2024-03-18T23:07:19.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.776+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f32a4061cf0 con 0x7f32900a29a0 2024-03-18T23:07:19.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.780+0000 7f329a7fc640 1 -- 172.21.15.132:0/4251882963 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f32a4061cf0 con 0x7f32900a29a0 2024-03-18T23:07:19.781 INFO:teuthology.orchestra.run.smithi132.stdout:/volumes/qa/sv_0/31591b33-9265-427a-a147-882001145179 2024-03-18T23:07:19.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 msgr2=0x7f32900a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 0x7f32900a4e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f329402c8e0 tx=0x7f3294005b50 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 msgr2=0x7f32a4197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a4197bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009b50 tx=0x7f32a000b690 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 shutdown_connections 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f32900a29a0 0x7f32900a4e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a4107640 0x7f32a419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a4104d10 0x7f32a4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 --2- 172.21.15.132:0/4251882963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32a41023e0 0x7f32a4197bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.782+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 >> 172.21.15.132:0/4251882963 conn(0x7f32a40fbd30 msgr2=0x7f32a4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.783+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 shutdown_connections 2024-03-18T23:07:19.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.783+0000 7f32ab922640 1 -- 172.21.15.132:0/4251882963 wait complete. 2024-03-18T23:07:19.794 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-18T23:07:19.859 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:19 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: 2024-03-18T23:07:19.475+0000 7f8976070700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T23:07:19.859 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:19 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: 2024-03-18T23:07:19.475+0000 7f8976070700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-18T23:07:19.859 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:07:19 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: 2024-03-18T23:07:19.475+0000 7f8976070700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 -- 172.21.15.132:0/1740413206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 msgr2=0x7f3d80101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 --2- 172.21.15.132:0/1740413206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d80101d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009900 tx=0x7f3d7002bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 -- 172.21.15.132:0/1740413206 shutdown_connections 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 --2- 172.21.15.132:0/1740413206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d801022d0 0x7f3d80108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 --2- 172.21.15.132:0/1740413206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d80101d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 --2- 172.21.15.132:0/1740413206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d80100720 0x7f3d80100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 -- 172.21.15.132:0/1740413206 >> 172.21.15.132:0/1740413206 conn(0x7f3d800fbcb0 msgr2=0x7f3d800fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 -- 172.21.15.132:0/1740413206 shutdown_connections 2024-03-18T23:07:19.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.904+0000 7f3d88485640 1 -- 172.21.15.132:0/1740413206 wait complete. 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 Processor -- start 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 -- start start 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d80100720 0x7f3d80069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d801022d0 0x7f3d8019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010c3d0 con 0x7f3d80101910 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010c250 con 0x7f3d801022d0 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d88485640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010c550 con 0x7f3d80100720 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49522/0 (socket says 172.21.15.132:49522) 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 -- 172.21.15.132:0/1585892642 learned_addr learned my addr 172.21.15.132:0/1585892642 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d86a22640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d801022d0 0x7f3d8019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d86221640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d80100720 0x7f3d80069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 -- 172.21.15.132:0/1585892642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d80100720 msgr2=0x7f3d80069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d80100720 0x7f3d80069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 -- 172.21.15.132:0/1585892642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d801022d0 msgr2=0x7f3d8019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d801022d0 0x7f3d8019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 -- 172.21.15.132:0/1585892642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8019f760 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d85a20640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d7002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.905+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70043020 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.906+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8019f9f0 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.906+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d700054e0 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.906+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70003c40 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.906+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8019ff40 con 0x7f3d80101910 2024-03-18T23:07:19.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.907+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d80107a90 con 0x7f3d80101910 2024-03-18T23:07:19.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.908+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3d70046020 con 0x7f3d80101910 2024-03-18T23:07:19.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.908+0000 7f3d777fe640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 0x7f3d6c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:19.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.908+0000 7f3d86221640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 0x7f3d6c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:19.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.909+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3d700f09a0 con 0x7f3d80101910 2024-03-18T23:07:19.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.909+0000 7f3d86221640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 0x7f3d6c0a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0095f0 tx=0x7f3d7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:19.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:19.914+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3d7004d020 con 0x7f3d80101910 2024-03-18T23:07:20.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-mon[32020]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-03-18T23:07:20.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:19 smithi132 ceph-mon[32020]: from='client.24586 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.115+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3d80061cf0 con 0x7f3d6c0a2890 2024-03-18T23:07:20.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.134+0000 7f3d777fe640 1 -- 172.21.15.132:0/1585892642 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d80061cf0 con 0x7f3d6c0a2890 2024-03-18T23:07:20.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 msgr2=0x7f3d6c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 0x7f3d6c0a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0095f0 tx=0x7f3d7c005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 msgr2=0x7f3d8019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d7002c650 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 shutdown_connections 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3d6c0a2890 0x7f3d6c0a4d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d801022d0 0x7f3d8019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d80101910 0x7f3d8019c830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 --2- 172.21.15.132:0/1585892642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d80100720 0x7f3d80069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 >> 172.21.15.132:0/1585892642 conn(0x7f3d800fbcb0 msgr2=0x7f3d80104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 shutdown_connections 2024-03-18T23:07:20.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.136+0000 7f3d88485640 1 -- 172.21.15.132:0/1585892642 wait complete. 2024-03-18T23:07:20.147 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-18T23:07:20.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:19 smithi196 ceph-mon[67089]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-03-18T23:07:20.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:19 smithi196 ceph-mon[67089]: from='client.24586 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:20.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.253+0000 7fb470359640 1 -- 172.21.15.132:0/3792129018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4681022a0 msgr2=0x7fb468104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.256 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.253+0000 7fb470359640 1 --2- 172.21.15.132:0/3792129018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4681022a0 0x7fb468104690 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb464009ca0 tx=0x7fb46402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- 172.21.15.132:0/3792129018 shutdown_connections 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- 172.21.15.132:0/3792129018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb468107500 0x7fb468109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- 172.21.15.132:0/3792129018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 0x7fb468106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- 172.21.15.132:0/3792129018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4681022a0 0x7fb468104690 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- 172.21.15.132:0/3792129018 >> 172.21.15.132:0/3792129018 conn(0x7fb4680fbcb0 msgr2=0x7fb4680fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- 172.21.15.132:0/3792129018 shutdown_connections 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- 172.21.15.132:0/3792129018 wait complete. 2024-03-18T23:07:20.257 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 Processor -- start 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- start start 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 0x7fb46819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb468107500 0x7fb4681a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb46810d4d0 con 0x7fb4681022a0 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb46810d350 con 0x7fb468104bd0 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb470359640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb46810d650 con 0x7fb468107500 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb46e0f5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb46e0f5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49542/0 (socket says 172.21.15.132:49542) 2024-03-18T23:07:20.258 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb46e0f5640 1 -- 172.21.15.132:0/1873787823 learned_addr learned my addr 172.21.15.132:0/1873787823 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb46e8f6640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb468107500 0x7fb4681a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.254+0000 7fb46d8f4640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 0x7fb46819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 -- 172.21.15.132:0/1873787823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb468107500 msgr2=0x7fb4681a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb468107500 0x7fb4681a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 -- 172.21.15.132:0/1873787823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 msgr2=0x7fb46819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 0x7fb46819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 -- 172.21.15.132:0/1873787823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681a19d0 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb46e0f5640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb464009dc0 tx=0x7fb464032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464041020 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681a1c60 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb464034cd0 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464039040 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.255+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681a21a0 con 0x7fb4681022a0 2024-03-18T23:07:20.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.256+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb468109050 con 0x7fb4681022a0 2024-03-18T23:07:20.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.257+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fb464044020 con 0x7fb4681022a0 2024-03-18T23:07:20.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.258+0000 7fb45f7fe640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 0x7fb4540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.260 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.258+0000 7fb46d8f4640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 0x7fb4540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.258+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb4640f0810 con 0x7fb4681022a0 2024-03-18T23:07:20.261 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.258+0000 7fb46d8f4640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 0x7fb4540a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4680694e0 tx=0x7fb458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.263+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb46404b020 con 0x7fb4681022a0 2024-03-18T23:07:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:19 smithi144 ceph-mon[66781]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-03-18T23:07:20.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:19 smithi144 ceph-mon[66781]: from='client.24586 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:20.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.465+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb468061cf0 con 0x7fb4540a2890 2024-03-18T23:07:20.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.469+0000 7fb45f7fe640 1 -- 172.21.15.132:0/1873787823 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb468061cf0 con 0x7fb4540a2890 2024-03-18T23:07:20.469 INFO:teuthology.orchestra.run.smithi132.stdout:/volumes/qa/sv_1/e3a84061-a703-4239-8f60-e71c218ee9e0 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 msgr2=0x7fb4540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 0x7fb4540a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4680694e0 tx=0x7fb458005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 msgr2=0x7fb468199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb464009dc0 tx=0x7fb464032f20 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 shutdown_connections 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb4540a2890 0x7fb4540a4d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb468107500 0x7fb4681a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb468104bd0 0x7fb46819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 --2- 172.21.15.132:0/1873787823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4681022a0 0x7fb468199dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 >> 172.21.15.132:0/1873787823 conn(0x7fb4680fbcb0 msgr2=0x7fb4681019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 shutdown_connections 2024-03-18T23:07:20.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.471+0000 7fb470359640 1 -- 172.21.15.132:0/1873787823 wait complete. 2024-03-18T23:07:20.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:07:20.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 -- 172.21.15.132:0/2628677076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 msgr2=0x7f3df0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.592 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2628677076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0102ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009860 tx=0x7f3de002bb70 comp rx=0 tx=0).stop 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 -- 172.21.15.132:0/2628677076 shutdown_connections 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2628677076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2628677076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0102ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2628677076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 0x7f3df0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.590+0000 7f3df87e6640 1 -- 172.21.15.132:0/2628677076 >> 172.21.15.132:0/2628677076 conn(0x7f3df00fce00 msgr2=0x7f3df00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- 172.21.15.132:0/2628677076 shutdown_connections 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- 172.21.15.132:0/2628677076 wait complete. 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 Processor -- start 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- start start 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 0x7f3df0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.593 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010d490 con 0x7f3df01033e0 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010d310 con 0x7f3df0102a20 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df87e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010d610 con 0x7f3df0101830 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df6d83640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df6d83640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49550/0 (socket says 172.21.15.132:49550) 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df6d83640 1 -- 172.21.15.132:0/2961816618 learned_addr learned my addr 172.21.15.132:0/2961816618 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df6582640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 0x7f3df0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.591+0000 7f3df5d81640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 -- 172.21.15.132:0/2961816618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 msgr2=0x7f3df0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 0x7f3df0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 -- 172.21.15.132:0/2961816618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 msgr2=0x7f3df0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 -- 172.21.15.132:0/2961816618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df019f720 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df6d83640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de800eb10 tx=0x7f3de800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8004b20 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df019fa10 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de8004cc0 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8010760 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.592+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df019ff30 con 0x7f3df01033e0 2024-03-18T23:07:20.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.593+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df0108b50 con 0x7f3df01033e0 2024-03-18T23:07:20.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.594+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3de8005020 con 0x7f3df01033e0 2024-03-18T23:07:20.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.594+0000 7f3de77fe640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 0x7f3ddc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.594+0000 7f3df6582640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 0x7f3ddc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.594+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3de80cefb0 con 0x7f3df01033e0 2024-03-18T23:07:20.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.595+0000 7f3df6582640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 0x7f3ddc0a4d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009530 tx=0x7f3dec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.599+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3de809c0a0 con 0x7f3df01033e0 2024-03-18T23:07:20.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.811+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3df002d1f0 con 0x7f3df01033e0 2024-03-18T23:07:20.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.812+0000 7f3de77fe640 1 -- 172.21.15.132:0/2961816618 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7f3de809b9a0 con 0x7f3df01033e0 2024-03-18T23:07:20.812 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:20.812 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:07:20.812 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 msgr2=0x7f3ddc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 0x7f3ddc0a4d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009530 tx=0x7f3dec009260 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 msgr2=0x7f3df019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de800eb10 tx=0x7f3de800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 shutdown_connections 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ddc0a2890 0x7f3ddc0a4d50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3df01033e0 0x7f3df019f0b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df0102a20 0x7f3df0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 --2- 172.21.15.132:0/2961816618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3df0101830 0x7f3df0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 >> 172.21.15.132:0/2961816618 conn(0x7f3df00fce00 msgr2=0x7f3df0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 shutdown_connections 2024-03-18T23:07:20.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.814+0000 7f3df87e6640 1 -- 172.21.15.132:0/2961816618 wait complete. 2024-03-18T23:07:20.825 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.929+0000 7f809833b640 1 -- 172.21.15.132:0/2411992517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80901022f0 msgr2=0x7f8090108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.929+0000 7f809833b640 1 --2- 172.21.15.132:0/2411992517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80901022f0 0x7f8090108b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8088009ca0 tx=0x7f808802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- 172.21.15.132:0/2411992517 shutdown_connections 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- 172.21.15.132:0/2411992517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80901022f0 0x7f8090108b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- 172.21.15.132:0/2411992517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 0x7f8090101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- 172.21.15.132:0/2411992517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8090100740 0x7f8090100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- 172.21.15.132:0/2411992517 >> 172.21.15.132:0/2411992517 conn(0x7f80900fbcb0 msgr2=0x7f80900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- 172.21.15.132:0/2411992517 shutdown_connections 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- 172.21.15.132:0/2411992517 wait complete. 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 Processor -- start 2024-03-18T23:07:20.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- start start 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 0x7f809019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80901022f0 0x7f809019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f809010c3c0 con 0x7f8090100740 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f809010c240 con 0x7f80901022f0 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f809833b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f809010c540 con 0x7f8090101930 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f80960d7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f80960d7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49568/0 (socket says 172.21.15.132:49568) 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.930+0000 7f80960d7640 1 -- 172.21.15.132:0/920376102 learned_addr learned my addr 172.21.15.132:0/920376102 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80958d6640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 0x7f809019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80968d8640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80901022f0 0x7f809019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 -- 172.21.15.132:0/920376102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 msgr2=0x7f809019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 0x7f809019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 -- 172.21.15.132:0/920376102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80901022f0 msgr2=0x7f809019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80901022f0 0x7f809019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 -- 172.21.15.132:0/920376102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809019f780 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80960d7640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9c0 tx=0x7f808c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00cd80 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c0035f0 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019fa10 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c010690 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.931+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809019ffb0 con 0x7f8090100740 2024-03-18T23:07:20.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.932+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8090107a60 con 0x7f8090100740 2024-03-18T23:07:20.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.933+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f808c0037a0 con 0x7f8090100740 2024-03-18T23:07:20.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.934+0000 7f80877fe640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 0x7f807c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:20.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.934+0000 7f80958d6640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 0x7f807c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:20.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.934+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f808c0cee40 con 0x7f8090100740 2024-03-18T23:07:20.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.934+0000 7f80958d6640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 0x7f807c0a4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8090069260 tx=0x7f8080005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:20.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:20.937+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f808c09bfc0 con 0x7f8090100740 2024-03-18T23:07:21.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.114+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f809002d3c0 con 0x7f8090100740 2024-03-18T23:07:21.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.115+0000 7f80877fe640 1 -- 172.21.15.132:0/920376102 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22266 (secure 0 0 0) 0x7f808c09b8c0 con 0x7f8090100740 2024-03-18T23:07:21.115 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:21.115 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:07:15.045318+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:21.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:20 smithi132 ceph-mon[32020]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:20 smithi132 ceph-mon[32020]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:20 smithi132 ceph-mon[32020]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-18T23:07:21.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:20 smithi132 ceph-mon[32020]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:20 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2961816618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:21.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 msgr2=0x7f807c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 0x7f807c0a4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8090069260 tx=0x7f8080005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:21.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 msgr2=0x7f80900694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9c0 tx=0x7f808c00ee80 comp rx=0 tx=0).stop 2024-03-18T23:07:21.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 shutdown_connections 2024-03-18T23:07:21.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f807c0a2910 0x7f807c0a4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80901022f0 0x7f809019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8090101930 0x7f809019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 --2- 172.21.15.132:0/920376102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8090100740 0x7f80900694f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 >> 172.21.15.132:0/920376102 conn(0x7f80900fbcb0 msgr2=0x7f8090104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 shutdown_connections 2024-03-18T23:07:21.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.117+0000 7f809833b640 1 -- 172.21.15.132:0/920376102 wait complete. 2024-03-18T23:07:21.129 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T23:07:21.129 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T23:07:21.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:20 smithi196 ceph-mon[67089]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:20 smithi196 ceph-mon[67089]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:20 smithi196 ceph-mon[67089]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-18T23:07:21.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:20 smithi196 ceph-mon[67089]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:20 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2961816618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:21.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 -- 172.21.15.132:0/368486727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 msgr2=0x7f1024108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 --2- 172.21.15.132:0/368486727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f1024108b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f101c009ca0 tx=0x7f101c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 -- 172.21.15.132:0/368486727 shutdown_connections 2024-03-18T23:07:21.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 --2- 172.21.15.132:0/368486727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f1024108b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 --2- 172.21.15.132:0/368486727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f1024101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 --2- 172.21.15.132:0/368486727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 0x7f1024100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.231+0000 7f1029f40640 1 -- 172.21.15.132:0/368486727 >> 172.21.15.132:0/368486727 conn(0x7f10240fbcb0 msgr2=0x7f10240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- 172.21.15.132:0/368486727 shutdown_connections 2024-03-18T23:07:21.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- 172.21.15.132:0/368486727 wait complete. 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 Processor -- start 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- start start 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 0x7f1024069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f102419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c3a0 con 0x7f10241022f0 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c220 con 0x7f1024100740 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1029f40640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c520 con 0x7f1024101930 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.232+0000 7f1023fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f10237fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 0x7f1024069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.235 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f102419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1023fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49584/0 (socket says 172.21.15.132:49584) 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1023fff640 1 -- 172.21.15.132:0/1220238021 learned_addr learned my addr 172.21.15.132:0/1220238021 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 -- 172.21.15.132:0/1220238021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 msgr2=0x7f1024069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 0x7f1024069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 -- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 msgr2=0x7f102419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 -- 172.21.15.132:0/1220238021 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102419f6f0 con 0x7f1024101930 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1023fff640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1022ffd640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f102419c7f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400cd80 con 0x7f1024101930 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102419f9e0 con 0x7f1024101930 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10140035f0 con 0x7f1024101930 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014010670 con 0x7f1024101930 2024-03-18T23:07:21.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.233+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102419ff80 con 0x7f1024101930 2024-03-18T23:07:21.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.234+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1024107a00 con 0x7f1024101930 2024-03-18T23:07:21.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.235+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f10140037a0 con 0x7f1024101930 2024-03-18T23:07:21.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.236+0000 7f1020ff9640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 0x7f100c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.236+0000 7f10237fe640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 0x7f100c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.236+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f10140ced00 con 0x7f1024101930 2024-03-18T23:07:21.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.236+0000 7f10237fe640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 0x7f100c0a4d50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1024101790 tx=0x7f1018005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.239 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.240+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f10140d2030 con 0x7f1024101930 2024-03-18T23:07:21.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:20 smithi144 ceph-mon[66781]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:20 smithi144 ceph-mon[66781]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:20 smithi144 ceph-mon[66781]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-03-18T23:07:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:20 smithi144 ceph-mon[66781]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:21.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:20 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2961816618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:07:21.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.413+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1024061c60 con 0x7f100c0a2890 2024-03-18T23:07:21.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.421+0000 7f1020ff9640 1 -- 172.21.15.132:0/1220238021 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258733 (secure 0 0 0) 0x7f1024061c60 con 0x7f100c0a2890 2024-03-18T23:07:21.425 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:21.426 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-18T23:07:21.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 msgr2=0x7f100c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 0x7f100c0a4d50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1024101790 tx=0x7f1018005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:21.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 msgr2=0x7f102419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f102419c7f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 shutdown_connections 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f100c0a2890 0x7f100c0a4d50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10241022f0 0x7f102419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1024101930 0x7f102419c7f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 --2- 172.21.15.132:0/1220238021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1024100740 0x7f1024069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 >> 172.21.15.132:0/1220238021 conn(0x7f10240fbcb0 msgr2=0x7f1024104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 shutdown_connections 2024-03-18T23:07:21.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.424+0000 7f1029f40640 1 -- 172.21.15.132:0/1220238021 wait complete. 2024-03-18T23:07:21.435 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-18T23:07:21.044559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614395,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":152,"num_read_kb":151,"num_write":295,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370548,"kb_used_data":13028,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571276,"statfs":{"total":1151940427776,"available":1151560986624,"internally_reserved":0,"allocated":13340672,"data_stored":7206566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24027,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":114,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.557927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106381+0000","last_change":"2024-03-18T23:07:14.066593+0000","last_active":"2024-03-18T23:07:15.106381+0000","last_peered":"2024-03-18T23:07:15.106381+0000","last_clean":"2024-03-18T23:07:15.106381+0000","last_became_active":"2024-03-18T23:07:14.066064+0000","last_became_peered":"2024-03-18T23:07:14.066064+0000","last_unstale":"2024-03-18T23:07:15.106381+0000","last_undegraded":"2024-03-18T23:07:15.106381+0000","last_fullsized":"2024-03-18T23:07:15.106381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:48.122520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048742+0000","last_change":"2024-03-18T23:07:14.070484+0000","last_active":"2024-03-18T23:07:15.048742+0000","last_peered":"2024-03-18T23:07:15.048742+0000","last_clean":"2024-03-18T23:07:15.048742+0000","last_became_active":"2024-03-18T23:07:14.070129+0000","last_became_peered":"2024-03-18T23:07:14.070129+0000","last_unstale":"2024-03-18T23:07:15.048742+0000","last_undegraded":"2024-03-18T23:07:15.048742+0000","last_fullsized":"2024-03-18T23:07:15.048742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:13:44.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048820+0000","last_change":"2024-03-18T23:07:14.064923+0000","last_active":"2024-03-18T23:07:15.048820+0000","last_peered":"2024-03-18T23:07:15.048820+0000","last_clean":"2024-03-18T23:07:15.048820+0000","last_became_active":"2024-03-18T23:07:14.064513+0000","last_became_peered":"2024-03-18T23:07:14.064513+0000","last_unstale":"2024-03-18T23:07:15.048820+0000","last_undegraded":"2024-03-18T23:07:15.048820+0000","last_fullsized":"2024-03-18T23:07:15.048820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:33.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105536+0000","last_change":"2024-03-18T23:07:14.065970+0000","last_active":"2024-03-18T23:07:15.105536+0000","last_peered":"2024-03-18T23:07:15.105536+0000","last_clean":"2024-03-18T23:07:15.105536+0000","last_became_active":"2024-03-18T23:07:14.065361+0000","last_became_peered":"2024-03-18T23:07:14.065361+0000","last_unstale":"2024-03-18T23:07:15.105536+0000","last_undegraded":"2024-03-18T23:07:15.105536+0000","last_fullsized":"2024-03-18T23:07:15.105536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:54:45.552015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105736+0000","last_change":"2024-03-18T23:07:14.058801+0000","last_active":"2024-03-18T23:07:15.105736+0000","last_peered":"2024-03-18T23:07:15.105736+0000","last_clean":"2024-03-18T23:07:15.105736+0000","last_became_active":"2024-03-18T23:07:14.058481+0000","last_became_peered":"2024-03-18T23:07:14.058481+0000","last_unstale":"2024-03-18T23:07:15.105736+0000","last_undegraded":"2024-03-18T23:07:15.105736+0000","last_fullsized":"2024-03-18T23:07:15.105736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:55.420890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049487+0000","last_change":"2024-03-18T23:07:14.067320+0000","last_active":"2024-03-18T23:07:15.049487+0000","last_peered":"2024-03-18T23:07:15.049487+0000","last_clean":"2024-03-18T23:07:15.049487+0000","last_became_active":"2024-03-18T23:07:14.066987+0000","last_became_peered":"2024-03-18T23:07:14.066987+0000","last_unstale":"2024-03-18T23:07:15.049487+0000","last_undegraded":"2024-03-18T23:07:15.049487+0000","last_fullsized":"2024-03-18T23:07:15.049487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:52:33.006026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846260+0000","last_change":"2024-03-18T23:07:14.065320+0000","last_active":"2024-03-18T23:07:15.846260+0000","last_peered":"2024-03-18T23:07:15.846260+0000","last_clean":"2024-03-18T23:07:15.846260+0000","last_became_active":"2024-03-18T23:07:14.064280+0000","last_became_peered":"2024-03-18T23:07:14.064280+0000","last_unstale":"2024-03-18T23:07:15.846260+0000","last_undegraded":"2024-03-18T23:07:15.846260+0000","last_fullsized":"2024-03-18T23:07:15.846260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:07.838038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104762+0000","last_change":"2024-03-18T23:07:14.071351+0000","last_active":"2024-03-18T23:07:15.104762+0000","last_peered":"2024-03-18T23:07:15.104762+0000","last_clean":"2024-03-18T23:07:15.104762+0000","last_became_active":"2024-03-18T23:07:14.071053+0000","last_became_peered":"2024-03-18T23:07:14.071053+0000","last_unstale":"2024-03-18T23:07:15.104762+0000","last_undegraded":"2024-03-18T23:07:15.104762+0000","last_fullsized":"2024-03-18T23:07:15.104762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:28.846769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105887+0000","last_change":"2024-03-18T23:07:14.072829+0000","last_active":"2024-03-18T23:07:15.105887+0000","last_peered":"2024-03-18T23:07:15.105887+0000","last_clean":"2024-03-18T23:07:15.105887+0000","last_became_active":"2024-03-18T23:07:14.072493+0000","last_became_peered":"2024-03-18T23:07:14.072493+0000","last_unstale":"2024-03-18T23:07:15.105887+0000","last_undegraded":"2024-03-18T23:07:15.105887+0000","last_fullsized":"2024-03-18T23:07:15.105887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:58:12.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049869+0000","last_change":"2024-03-18T23:07:14.067119+0000","last_active":"2024-03-18T23:07:15.049869+0000","last_peered":"2024-03-18T23:07:15.049869+0000","last_clean":"2024-03-18T23:07:15.049869+0000","last_became_active":"2024-03-18T23:07:14.066738+0000","last_became_peered":"2024-03-18T23:07:14.066738+0000","last_unstale":"2024-03-18T23:07:15.049869+0000","last_undegraded":"2024-03-18T23:07:15.049869+0000","last_fullsized":"2024-03-18T23:07:15.049869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:55.011692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846831+0000","last_change":"2024-03-18T23:07:14.071791+0000","last_active":"2024-03-18T23:07:15.846831+0000","last_peered":"2024-03-18T23:07:15.846831+0000","last_clean":"2024-03-18T23:07:15.846831+0000","last_became_active":"2024-03-18T23:07:14.068031+0000","last_became_peered":"2024-03-18T23:07:14.068031+0000","last_unstale":"2024-03-18T23:07:15.846831+0000","last_undegraded":"2024-03-18T23:07:15.846831+0000","last_fullsized":"2024-03-18T23:07:15.846831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:42:56.142512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105994+0000","last_change":"2024-03-18T23:07:14.071712+0000","last_active":"2024-03-18T23:07:15.105994+0000","last_peered":"2024-03-18T23:07:15.105994+0000","last_clean":"2024-03-18T23:07:15.105994+0000","last_became_active":"2024-03-18T23:07:14.069595+0000","last_became_peered":"2024-03-18T23:07:14.069595+0000","last_unstale":"2024-03-18T23:07:15.105994+0000","last_undegraded":"2024-03-18T23:07:15.105994+0000","last_fullsized":"2024-03-18T23:07:15.105994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:10:21.521681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091828+0000","last_change":"2024-03-18T23:07:14.069861+0000","last_active":"2024-03-18T23:07:15.091828+0000","last_peered":"2024-03-18T23:07:15.091828+0000","last_clean":"2024-03-18T23:07:15.091828+0000","last_became_active":"2024-03-18T23:07:14.069534+0000","last_became_peered":"2024-03-18T23:07:14.069534+0000","last_unstale":"2024-03-18T23:07:15.091828+0000","last_undegraded":"2024-03-18T23:07:15.091828+0000","last_fullsized":"2024-03-18T23:07:15.091828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:14:51.360640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048494+0000","last_change":"2024-03-18T23:07:14.070755+0000","last_active":"2024-03-18T23:07:15.048494+0000","last_peered":"2024-03-18T23:07:15.048494+0000","last_clean":"2024-03-18T23:07:15.048494+0000","last_became_active":"2024-03-18T23:07:14.070126+0000","last_became_peered":"2024-03-18T23:07:14.070126+0000","last_unstale":"2024-03-18T23:07:15.048494+0000","last_undegraded":"2024-03-18T23:07:15.048494+0000","last_fullsized":"2024-03-18T23:07:15.048494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:05.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048550+0000","last_change":"2024-03-18T23:07:14.070166+0000","last_active":"2024-03-18T23:07:15.048550+0000","last_peered":"2024-03-18T23:07:15.048550+0000","last_clean":"2024-03-18T23:07:15.048550+0000","last_became_active":"2024-03-18T23:07:14.069693+0000","last_became_peered":"2024-03-18T23:07:14.069693+0000","last_unstale":"2024-03-18T23:07:15.048550+0000","last_undegraded":"2024-03-18T23:07:15.048550+0000","last_fullsized":"2024-03-18T23:07:15.048550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:55.616894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105260+0000","last_change":"2024-03-18T23:07:14.076732+0000","last_active":"2024-03-18T23:07:15.105260+0000","last_peered":"2024-03-18T23:07:15.105260+0000","last_clean":"2024-03-18T23:07:15.105260+0000","last_became_active":"2024-03-18T23:07:14.076143+0000","last_became_peered":"2024-03-18T23:07:14.076143+0000","last_unstale":"2024-03-18T23:07:15.105260+0000","last_undegraded":"2024-03-18T23:07:15.105260+0000","last_fullsized":"2024-03-18T23:07:15.105260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:46.888524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106761+0000","last_change":"2024-03-18T23:07:14.066388+0000","last_active":"2024-03-18T23:07:15.106761+0000","last_peered":"2024-03-18T23:07:15.106761+0000","last_clean":"2024-03-18T23:07:15.106761+0000","last_became_active":"2024-03-18T23:07:14.065626+0000","last_became_peered":"2024-03-18T23:07:14.065626+0000","last_unstale":"2024-03-18T23:07:15.106761+0000","last_undegraded":"2024-03-18T23:07:15.106761+0000","last_fullsized":"2024-03-18T23:07:15.106761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:08:18.200855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049523+0000","last_change":"2024-03-18T23:07:14.068730+0000","last_active":"2024-03-18T23:07:15.049523+0000","last_peered":"2024-03-18T23:07:15.049523+0000","last_clean":"2024-03-18T23:07:15.049523+0000","last_became_active":"2024-03-18T23:07:14.068420+0000","last_became_peered":"2024-03-18T23:07:14.068420+0000","last_unstale":"2024-03-18T23:07:15.049523+0000","last_undegraded":"2024-03-18T23:07:15.049523+0000","last_fullsized":"2024-03-18T23:07:15.049523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:41:34.588823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106361+0000","last_change":"2024-03-18T23:07:14.061415+0000","last_active":"2024-03-18T23:07:15.106361+0000","last_peered":"2024-03-18T23:07:15.106361+0000","last_clean":"2024-03-18T23:07:15.106361+0000","last_became_active":"2024-03-18T23:07:14.061102+0000","last_became_peered":"2024-03-18T23:07:14.061102+0000","last_unstale":"2024-03-18T23:07:15.106361+0000","last_undegraded":"2024-03-18T23:07:15.106361+0000","last_fullsized":"2024-03-18T23:07:15.106361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:06:08.056213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049174+0000","last_change":"2024-03-18T23:07:13.506404+0000","last_active":"2024-03-18T23:07:15.049174+0000","last_peered":"2024-03-18T23:07:15.049174+0000","last_clean":"2024-03-18T23:07:15.049174+0000","last_became_active":"2024-03-18T23:07:13.506091+0000","last_became_peered":"2024-03-18T23:07:13.506091+0000","last_unstale":"2024-03-18T23:07:15.049174+0000","last_undegraded":"2024-03-18T23:07:15.049174+0000","last_fullsized":"2024-03-18T23:07:15.049174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:34:26.916813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105056+0000","last_change":"2024-03-18T23:07:14.061990+0000","last_active":"2024-03-18T23:07:15.105056+0000","last_peered":"2024-03-18T23:07:15.105056+0000","last_clean":"2024-03-18T23:07:15.105056+0000","last_became_active":"2024-03-18T23:07:14.061581+0000","last_became_peered":"2024-03-18T23:07:14.061581+0000","last_unstale":"2024-03-18T23:07:15.105056+0000","last_undegraded":"2024-03-18T23:07:15.105056+0000","last_fullsized":"2024-03-18T23:07:15.105056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:37.726342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105077+0000","last_change":"2024-03-18T23:07:13.509727+0000","last_active":"2024-03-18T23:07:15.105077+0000","last_peered":"2024-03-18T23:07:15.105077+0000","last_clean":"2024-03-18T23:07:15.105077+0000","last_became_active":"2024-03-18T23:07:13.509296+0000","last_became_peered":"2024-03-18T23:07:13.509296+0000","last_unstale":"2024-03-18T23:07:15.105077+0000","last_undegraded":"2024-03-18T23:07:15.105077+0000","last_fullsized":"2024-03-18T23:07:15.105077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:15.060387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049561+0000","last_change":"2024-03-18T23:07:14.066038+0000","last_active":"2024-03-18T23:07:15.049561+0000","last_peered":"2024-03-18T23:07:15.049561+0000","last_clean":"2024-03-18T23:07:15.049561+0000","last_became_active":"2024-03-18T23:07:14.065449+0000","last_became_peered":"2024-03-18T23:07:14.065449+0000","last_unstale":"2024-03-18T23:07:15.049561+0000","last_undegraded":"2024-03-18T23:07:15.049561+0000","last_fullsized":"2024-03-18T23:07:15.049561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:53:07.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867050+0000","last_change":"2024-03-18T23:07:13.503422+0000","last_active":"2024-03-18T23:07:17.867050+0000","last_peered":"2024-03-18T23:07:17.867050+0000","last_clean":"2024-03-18T23:07:17.867050+0000","last_became_active":"2024-03-18T23:07:13.503068+0000","last_became_peered":"2024-03-18T23:07:13.503068+0000","last_unstale":"2024-03-18T23:07:17.867050+0000","last_undegraded":"2024-03-18T23:07:17.867050+0000","last_fullsized":"2024-03-18T23:07:17.867050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:35.369317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105538+0000","last_change":"2024-03-18T23:07:14.073092+0000","last_active":"2024-03-18T23:07:15.105538+0000","last_peered":"2024-03-18T23:07:15.105538+0000","last_clean":"2024-03-18T23:07:15.105538+0000","last_became_active":"2024-03-18T23:07:14.070303+0000","last_became_peered":"2024-03-18T23:07:14.070303+0000","last_unstale":"2024-03-18T23:07:15.105538+0000","last_undegraded":"2024-03-18T23:07:15.105538+0000","last_fullsized":"2024-03-18T23:07:15.105538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:09.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106947+0000","last_change":"2024-03-18T23:07:13.505420+0000","last_active":"2024-03-18T23:07:15.106947+0000","last_peered":"2024-03-18T23:07:15.106947+0000","last_clean":"2024-03-18T23:07:15.106947+0000","last_became_active":"2024-03-18T23:07:13.504995+0000","last_became_peered":"2024-03-18T23:07:13.504995+0000","last_unstale":"2024-03-18T23:07:15.106947+0000","last_undegraded":"2024-03-18T23:07:15.106947+0000","last_fullsized":"2024-03-18T23:07:15.106947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:50.988318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107146+0000","last_change":"2024-03-18T23:07:14.071254+0000","last_active":"2024-03-18T23:07:15.107146+0000","last_peered":"2024-03-18T23:07:15.107146+0000","last_clean":"2024-03-18T23:07:15.107146+0000","last_became_active":"2024-03-18T23:07:14.070767+0000","last_became_peered":"2024-03-18T23:07:14.070767+0000","last_unstale":"2024-03-18T23:07:15.107146+0000","last_undegraded":"2024-03-18T23:07:15.107146+0000","last_fullsized":"2024-03-18T23:07:15.107146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:36.639093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.866654+0000","last_change":"2024-03-18T23:07:13.505020+0000","last_active":"2024-03-18T23:07:16.866654+0000","last_peered":"2024-03-18T23:07:16.866654+0000","last_clean":"2024-03-18T23:07:16.866654+0000","last_became_active":"2024-03-18T23:07:13.504355+0000","last_became_peered":"2024-03-18T23:07:13.504355+0000","last_unstale":"2024-03-18T23:07:16.866654+0000","last_undegraded":"2024-03-18T23:07:16.866654+0000","last_fullsized":"2024-03-18T23:07:16.866654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:03.184513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105913+0000","last_change":"2024-03-18T23:07:14.063156+0000","last_active":"2024-03-18T23:07:15.105913+0000","last_peered":"2024-03-18T23:07:15.105913+0000","last_clean":"2024-03-18T23:07:15.105913+0000","last_became_active":"2024-03-18T23:07:14.062801+0000","last_became_peered":"2024-03-18T23:07:14.062801+0000","last_unstale":"2024-03-18T23:07:15.105913+0000","last_undegraded":"2024-03-18T23:07:15.105913+0000","last_fullsized":"2024-03-18T23:07:15.105913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:49:27.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867406+0000","last_change":"2024-03-18T23:07:13.501538+0000","last_active":"2024-03-18T23:07:17.867406+0000","last_peered":"2024-03-18T23:07:17.867406+0000","last_clean":"2024-03-18T23:07:17.867406+0000","last_became_active":"2024-03-18T23:07:13.501160+0000","last_became_peered":"2024-03-18T23:07:13.501160+0000","last_unstale":"2024-03-18T23:07:17.867406+0000","last_undegraded":"2024-03-18T23:07:17.867406+0000","last_fullsized":"2024-03-18T23:07:17.867406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.319636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106576+0000","last_change":"2024-03-18T23:07:14.069345+0000","last_active":"2024-03-18T23:07:15.106576+0000","last_peered":"2024-03-18T23:07:15.106576+0000","last_clean":"2024-03-18T23:07:15.106576+0000","last_became_active":"2024-03-18T23:07:14.068320+0000","last_became_peered":"2024-03-18T23:07:14.068320+0000","last_unstale":"2024-03-18T23:07:15.106576+0000","last_undegraded":"2024-03-18T23:07:15.106576+0000","last_fullsized":"2024-03-18T23:07:15.106576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:36.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091611+0000","last_change":"2024-03-18T23:07:13.507538+0000","last_active":"2024-03-18T23:07:15.091611+0000","last_peered":"2024-03-18T23:07:15.091611+0000","last_clean":"2024-03-18T23:07:15.091611+0000","last_became_active":"2024-03-18T23:07:13.506563+0000","last_became_peered":"2024-03-18T23:07:13.506563+0000","last_unstale":"2024-03-18T23:07:15.091611+0000","last_undegraded":"2024-03-18T23:07:15.091611+0000","last_fullsized":"2024-03-18T23:07:15.091611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:58.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104704+0000","last_change":"2024-03-18T23:07:14.062563+0000","last_active":"2024-03-18T23:07:15.104704+0000","last_peered":"2024-03-18T23:07:15.104704+0000","last_clean":"2024-03-18T23:07:15.104704+0000","last_became_active":"2024-03-18T23:07:14.062232+0000","last_became_peered":"2024-03-18T23:07:14.062232+0000","last_unstale":"2024-03-18T23:07:15.104704+0000","last_undegraded":"2024-03-18T23:07:15.104704+0000","last_fullsized":"2024-03-18T23:07:15.104704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:06.055127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864872+0000","last_change":"2024-03-18T23:07:13.505103+0000","last_active":"2024-03-18T23:07:16.864872+0000","last_peered":"2024-03-18T23:07:16.864872+0000","last_clean":"2024-03-18T23:07:16.864872+0000","last_became_active":"2024-03-18T23:07:13.504764+0000","last_became_peered":"2024-03-18T23:07:13.504764+0000","last_unstale":"2024-03-18T23:07:16.864872+0000","last_undegraded":"2024-03-18T23:07:16.864872+0000","last_fullsized":"2024-03-18T23:07:16.864872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:07:44.998932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049419+0000","last_change":"2024-03-18T23:07:14.066440+0000","last_active":"2024-03-18T23:07:15.049419+0000","last_peered":"2024-03-18T23:07:15.049419+0000","last_clean":"2024-03-18T23:07:15.049419+0000","last_became_active":"2024-03-18T23:07:14.066100+0000","last_became_peered":"2024-03-18T23:07:14.066100+0000","last_unstale":"2024-03-18T23:07:15.049419+0000","last_undegraded":"2024-03-18T23:07:15.049419+0000","last_fullsized":"2024-03-18T23:07:15.049419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:56:27.662447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.745939+0000","last_change":"2024-03-18T23:07:13.501952+0000","last_active":"2024-03-18T23:07:18.745939+0000","last_peered":"2024-03-18T23:07:18.745939+0000","last_clean":"2024-03-18T23:07:18.745939+0000","last_became_active":"2024-03-18T23:07:13.501628+0000","last_became_peered":"2024-03-18T23:07:13.501628+0000","last_unstale":"2024-03-18T23:07:18.745939+0000","last_undegraded":"2024-03-18T23:07:18.745939+0000","last_fullsized":"2024-03-18T23:07:18.745939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:09.046133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105985+0000","last_change":"2024-03-18T23:07:14.075917+0000","last_active":"2024-03-18T23:07:15.105985+0000","last_peered":"2024-03-18T23:07:15.105985+0000","last_clean":"2024-03-18T23:07:15.105985+0000","last_became_active":"2024-03-18T23:07:14.075522+0000","last_became_peered":"2024-03-18T23:07:14.075522+0000","last_unstale":"2024-03-18T23:07:15.105985+0000","last_undegraded":"2024-03-18T23:07:15.105985+0000","last_fullsized":"2024-03-18T23:07:15.105985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:28.108526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864211+0000","last_change":"2024-03-18T23:07:13.507271+0000","last_active":"2024-03-18T23:07:16.864211+0000","last_peered":"2024-03-18T23:07:16.864211+0000","last_clean":"2024-03-18T23:07:16.864211+0000","last_became_active":"2024-03-18T23:07:13.506771+0000","last_became_peered":"2024-03-18T23:07:13.506771+0000","last_unstale":"2024-03-18T23:07:16.864211+0000","last_undegraded":"2024-03-18T23:07:16.864211+0000","last_fullsized":"2024-03-18T23:07:16.864211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:23:08.552922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049435+0000","last_change":"2024-03-18T23:07:14.066838+0000","last_active":"2024-03-18T23:07:15.049435+0000","last_peered":"2024-03-18T23:07:15.049435+0000","last_clean":"2024-03-18T23:07:15.049435+0000","last_became_active":"2024-03-18T23:07:14.066550+0000","last_became_peered":"2024-03-18T23:07:14.066550+0000","last_unstale":"2024-03-18T23:07:15.049435+0000","last_undegraded":"2024-03-18T23:07:15.049435+0000","last_fullsized":"2024-03-18T23:07:15.049435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:59:19.048074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863902+0000","last_change":"2024-03-18T23:07:13.511861+0000","last_active":"2024-03-18T23:07:16.863902+0000","last_peered":"2024-03-18T23:07:16.863902+0000","last_clean":"2024-03-18T23:07:16.863902+0000","last_became_active":"2024-03-18T23:07:13.511458+0000","last_became_peered":"2024-03-18T23:07:13.511458+0000","last_unstale":"2024-03-18T23:07:16.863902+0000","last_undegraded":"2024-03-18T23:07:16.863902+0000","last_fullsized":"2024-03-18T23:07:16.863902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:03.301572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105849+0000","last_change":"2024-03-18T23:07:14.073392+0000","last_active":"2024-03-18T23:07:15.105849+0000","last_peered":"2024-03-18T23:07:15.105849+0000","last_clean":"2024-03-18T23:07:15.105849+0000","last_became_active":"2024-03-18T23:07:14.071962+0000","last_became_peered":"2024-03-18T23:07:14.071962+0000","last_unstale":"2024-03-18T23:07:15.105849+0000","last_undegraded":"2024-03-18T23:07:15.105849+0000","last_fullsized":"2024-03-18T23:07:15.105849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:55:01.175354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:19.749139+0000","last_change":"2024-03-18T23:07:13.505399+0000","last_active":"2024-03-18T23:07:19.749139+0000","last_peered":"2024-03-18T23:07:19.749139+0000","last_clean":"2024-03-18T23:07:19.749139+0000","last_became_active":"2024-03-18T23:07:13.505034+0000","last_became_peered":"2024-03-18T23:07:13.505034+0000","last_unstale":"2024-03-18T23:07:19.749139+0000","last_undegraded":"2024-03-18T23:07:19.749139+0000","last_fullsized":"2024-03-18T23:07:19.749139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:52.665189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106750+0000","last_change":"2024-03-18T23:07:14.059215+0000","last_active":"2024-03-18T23:07:15.106750+0000","last_peered":"2024-03-18T23:07:15.106750+0000","last_clean":"2024-03-18T23:07:15.106750+0000","last_became_active":"2024-03-18T23:07:14.058904+0000","last_became_peered":"2024-03-18T23:07:14.058904+0000","last_unstale":"2024-03-18T23:07:15.106750+0000","last_undegraded":"2024-03-18T23:07:15.106750+0000","last_fullsized":"2024-03-18T23:07:15.106750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:28:38.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.869100+0000","last_change":"2024-03-18T23:07:13.503163+0000","last_active":"2024-03-18T23:07:17.869100+0000","last_peered":"2024-03-18T23:07:17.869100+0000","last_clean":"2024-03-18T23:07:17.869100+0000","last_became_active":"2024-03-18T23:07:13.502816+0000","last_became_peered":"2024-03-18T23:07:13.502816+0000","last_unstale":"2024-03-18T23:07:17.869100+0000","last_undegraded":"2024-03-18T23:07:17.869100+0000","last_fullsized":"2024-03-18T23:07:17.869100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:38:23.950178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091679+0000","last_change":"2024-03-18T23:07:14.057680+0000","last_active":"2024-03-18T23:07:15.091679+0000","last_peered":"2024-03-18T23:07:15.091679+0000","last_clean":"2024-03-18T23:07:15.091679+0000","last_became_active":"2024-03-18T23:07:14.057344+0000","last_became_peered":"2024-03-18T23:07:14.057344+0000","last_unstale":"2024-03-18T23:07:15.091679+0000","last_undegraded":"2024-03-18T23:07:15.091679+0000","last_fullsized":"2024-03-18T23:07:15.091679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:40.616782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864726+0000","last_change":"2024-03-18T23:07:13.500994+0000","last_active":"2024-03-18T23:07:16.864726+0000","last_peered":"2024-03-18T23:07:16.864726+0000","last_clean":"2024-03-18T23:07:16.864726+0000","last_became_active":"2024-03-18T23:07:13.500678+0000","last_became_peered":"2024-03-18T23:07:13.500678+0000","last_unstale":"2024-03-18T23:07:16.864726+0000","last_undegraded":"2024-03-18T23:07:16.864726+0000","last_fullsized":"2024-03-18T23:07:16.864726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:41:30.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846633+0000","last_change":"2024-03-18T23:07:14.073945+0000","last_active":"2024-03-18T23:07:15.846633+0000","last_peered":"2024-03-18T23:07:15.846633+0000","last_clean":"2024-03-18T23:07:15.846633+0000","last_became_active":"2024-03-18T23:07:14.073561+0000","last_became_peered":"2024-03-18T23:07:14.073561+0000","last_unstale":"2024-03-18T23:07:15.846633+0000","last_undegraded":"2024-03-18T23:07:15.846633+0000","last_fullsized":"2024-03-18T23:07:15.846633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:33:26.514969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105115+0000","last_change":"2024-03-18T23:07:13.502121+0000","last_active":"2024-03-18T23:07:15.105115+0000","last_peered":"2024-03-18T23:07:15.105115+0000","last_clean":"2024-03-18T23:07:15.105115+0000","last_became_active":"2024-03-18T23:07:13.501796+0000","last_became_peered":"2024-03-18T23:07:13.501796+0000","last_unstale":"2024-03-18T23:07:15.105115+0000","last_undegraded":"2024-03-18T23:07:15.105115+0000","last_fullsized":"2024-03-18T23:07:15.105115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:02.348416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846134+0000","last_change":"2024-03-18T23:07:14.074062+0000","last_active":"2024-03-18T23:07:15.846134+0000","last_peered":"2024-03-18T23:07:15.846134+0000","last_clean":"2024-03-18T23:07:15.846134+0000","last_became_active":"2024-03-18T23:07:14.073657+0000","last_became_peered":"2024-03-18T23:07:14.073657+0000","last_unstale":"2024-03-18T23:07:15.846134+0000","last_undegraded":"2024-03-18T23:07:15.846134+0000","last_fullsized":"2024-03-18T23:07:15.846134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:28.103535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.868925+0000","last_change":"2024-03-18T23:07:13.509334+0000","last_active":"2024-03-18T23:07:17.868925+0000","last_peered":"2024-03-18T23:07:17.868925+0000","last_clean":"2024-03-18T23:07:17.868925+0000","last_became_active":"2024-03-18T23:07:13.509036+0000","last_became_peered":"2024-03-18T23:07:13.509036+0000","last_unstale":"2024-03-18T23:07:17.868925+0000","last_undegraded":"2024-03-18T23:07:17.868925+0000","last_fullsized":"2024-03-18T23:07:17.868925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:14:27.935984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107718+0000","last_change":"2024-03-18T23:07:14.068751+0000","last_active":"2024-03-18T23:07:15.107718+0000","last_peered":"2024-03-18T23:07:15.107718+0000","last_clean":"2024-03-18T23:07:15.107718+0000","last_became_active":"2024-03-18T23:07:14.068417+0000","last_became_peered":"2024-03-18T23:07:14.068417+0000","last_unstale":"2024-03-18T23:07:15.107718+0000","last_undegraded":"2024-03-18T23:07:15.107718+0000","last_fullsized":"2024-03-18T23:07:15.107718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:26:31.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.745023+0000","last_change":"2024-03-18T23:07:13.499616+0000","last_active":"2024-03-18T23:07:18.745023+0000","last_peered":"2024-03-18T23:07:18.745023+0000","last_clean":"2024-03-18T23:07:18.745023+0000","last_became_active":"2024-03-18T23:07:13.499129+0000","last_became_peered":"2024-03-18T23:07:13.499129+0000","last_unstale":"2024-03-18T23:07:18.745023+0000","last_undegraded":"2024-03-18T23:07:18.745023+0000","last_fullsized":"2024-03-18T23:07:18.745023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:28.699813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106374+0000","last_change":"2024-03-18T23:07:14.066305+0000","last_active":"2024-03-18T23:07:15.106374+0000","last_peered":"2024-03-18T23:07:15.106374+0000","last_clean":"2024-03-18T23:07:15.106374+0000","last_became_active":"2024-03-18T23:07:14.065952+0000","last_became_peered":"2024-03-18T23:07:14.065952+0000","last_unstale":"2024-03-18T23:07:15.106374+0000","last_undegraded":"2024-03-18T23:07:15.106374+0000","last_fullsized":"2024-03-18T23:07:15.106374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:06:00.718689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049291+0000","last_change":"2024-03-18T23:07:13.508496+0000","last_active":"2024-03-18T23:07:15.049291+0000","last_peered":"2024-03-18T23:07:15.049291+0000","last_clean":"2024-03-18T23:07:15.049291+0000","last_became_active":"2024-03-18T23:07:13.508041+0000","last_became_peered":"2024-03-18T23:07:13.508041+0000","last_unstale":"2024-03-18T23:07:15.049291+0000","last_undegraded":"2024-03-18T23:07:15.049291+0000","last_fullsized":"2024-03-18T23:07:15.049291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:22.989161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.102627+0000","last_change":"2024-03-18T23:07:14.072979+0000","last_active":"2024-03-18T23:07:15.102627+0000","last_peered":"2024-03-18T23:07:15.102627+0000","last_clean":"2024-03-18T23:07:15.102627+0000","last_became_active":"2024-03-18T23:07:14.072663+0000","last_became_peered":"2024-03-18T23:07:14.072663+0000","last_unstale":"2024-03-18T23:07:15.102627+0000","last_undegraded":"2024-03-18T23:07:15.102627+0000","last_fullsized":"2024-03-18T23:07:15.102627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:19.916681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.959996+0000","last_change":"2024-03-18T23:07:13.506789+0000","last_active":"2024-03-18T23:07:15.959996+0000","last_peered":"2024-03-18T23:07:15.959996+0000","last_clean":"2024-03-18T23:07:15.959996+0000","last_became_active":"2024-03-18T23:07:13.506412+0000","last_became_peered":"2024-03-18T23:07:13.506412+0000","last_unstale":"2024-03-18T23:07:15.959996+0000","last_undegraded":"2024-03-18T23:07:15.959996+0000","last_fullsized":"2024-03-18T23:07:15.959996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:37.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049560+0000","last_change":"2024-03-18T23:07:14.071419+0000","last_active":"2024-03-18T23:07:15.049560+0000","last_peered":"2024-03-18T23:07:15.049560+0000","last_clean":"2024-03-18T23:07:15.049560+0000","last_became_active":"2024-03-18T23:07:14.071071+0000","last_became_peered":"2024-03-18T23:07:14.071071+0000","last_unstale":"2024-03-18T23:07:15.049560+0000","last_undegraded":"2024-03-18T23:07:15.049560+0000","last_fullsized":"2024-03-18T23:07:15.049560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:25.657312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107514+0000","last_change":"2024-03-18T23:07:14.065418+0000","last_active":"2024-03-18T23:07:15.107514+0000","last_peered":"2024-03-18T23:07:15.107514+0000","last_clean":"2024-03-18T23:07:15.107514+0000","last_became_active":"2024-03-18T23:07:14.065057+0000","last_became_peered":"2024-03-18T23:07:14.065057+0000","last_unstale":"2024-03-18T23:07:15.107514+0000","last_undegraded":"2024-03-18T23:07:15.107514+0000","last_fullsized":"2024-03-18T23:07:15.107514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:23:29.531861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091513+0000","last_change":"2024-03-18T23:07:13.510913+0000","last_active":"2024-03-18T23:07:15.091513+0000","last_peered":"2024-03-18T23:07:15.091513+0000","last_clean":"2024-03-18T23:07:15.091513+0000","last_became_active":"2024-03-18T23:07:13.510563+0000","last_became_peered":"2024-03-18T23:07:13.510563+0000","last_unstale":"2024-03-18T23:07:15.091513+0000","last_undegraded":"2024-03-18T23:07:15.091513+0000","last_fullsized":"2024-03-18T23:07:15.091513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:15.798492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105244+0000","last_change":"2024-03-18T23:07:14.065203+0000","last_active":"2024-03-18T23:07:15.105244+0000","last_peered":"2024-03-18T23:07:15.105244+0000","last_clean":"2024-03-18T23:07:15.105244+0000","last_became_active":"2024-03-18T23:07:14.064924+0000","last_became_peered":"2024-03-18T23:07:14.064924+0000","last_unstale":"2024-03-18T23:07:15.105244+0000","last_undegraded":"2024-03-18T23:07:15.105244+0000","last_fullsized":"2024-03-18T23:07:15.105244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:32:11.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863077+0000","last_change":"2024-03-18T23:07:13.495244+0000","last_active":"2024-03-18T23:07:16.863077+0000","last_peered":"2024-03-18T23:07:16.863077+0000","last_clean":"2024-03-18T23:07:16.863077+0000","last_became_active":"2024-03-18T23:07:13.494879+0000","last_became_peered":"2024-03-18T23:07:13.494879+0000","last_unstale":"2024-03-18T23:07:16.863077+0000","last_undegraded":"2024-03-18T23:07:16.863077+0000","last_fullsized":"2024-03-18T23:07:16.863077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:14:51.684956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846102+0000","last_change":"2024-03-18T23:06:03.931253+0000","last_active":"2024-03-18T23:07:15.846102+0000","last_peered":"2024-03-18T23:07:15.846102+0000","last_clean":"2024-03-18T23:07:15.846102+0000","last_became_active":"2024-03-18T23:06:03.930687+0000","last_became_peered":"2024-03-18T23:06:03.930687+0000","last_unstale":"2024-03-18T23:07:15.846102+0000","last_undegraded":"2024-03-18T23:07:15.846102+0000","last_fullsized":"2024-03-18T23:07:15.846102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_clean_scrub_stamp":"2024-03-18T23:02:45.345904+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:08:47.199331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091856+0000","last_change":"2024-03-18T23:07:14.065457+0000","last_active":"2024-03-18T23:07:15.091856+0000","last_peered":"2024-03-18T23:07:15.091856+0000","last_clean":"2024-03-18T23:07:15.091856+0000","last_became_active":"2024-03-18T23:07:14.065117+0000","last_became_peered":"2024-03-18T23:07:14.065117+0000","last_unstale":"2024-03-18T23:07:15.091856+0000","last_undegraded":"2024-03-18T23:07:15.091856+0000","last_fullsized":"2024-03-18T23:07:15.091856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:43.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.868349+0000","last_change":"2024-03-18T23:07:13.501074+0000","last_active":"2024-03-18T23:07:17.868349+0000","last_peered":"2024-03-18T23:07:17.868349+0000","last_clean":"2024-03-18T23:07:17.868349+0000","last_became_active":"2024-03-18T23:07:13.500549+0000","last_became_peered":"2024-03-18T23:07:13.500549+0000","last_unstale":"2024-03-18T23:07:17.868349+0000","last_undegraded":"2024-03-18T23:07:17.868349+0000","last_fullsized":"2024-03-18T23:07:17.868349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:53:04.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863994+0000","last_change":"2024-03-18T23:07:13.510215+0000","last_active":"2024-03-18T23:07:16.863994+0000","last_peered":"2024-03-18T23:07:16.863994+0000","last_clean":"2024-03-18T23:07:16.863994+0000","last_became_active":"2024-03-18T23:07:13.509864+0000","last_became_peered":"2024-03-18T23:07:13.509864+0000","last_unstale":"2024-03-18T23:07:16.863994+0000","last_undegraded":"2024-03-18T23:07:16.863994+0000","last_fullsized":"2024-03-18T23:07:16.863994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:00:35.421397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091747+0000","last_change":"2024-03-18T23:07:14.058971+0000","last_active":"2024-03-18T23:07:15.091747+0000","last_peered":"2024-03-18T23:07:15.091747+0000","last_clean":"2024-03-18T23:07:15.091747+0000","last_became_active":"2024-03-18T23:07:14.058619+0000","last_became_peered":"2024-03-18T23:07:14.058619+0000","last_unstale":"2024-03-18T23:07:15.091747+0000","last_undegraded":"2024-03-18T23:07:15.091747+0000","last_fullsized":"2024-03-18T23:07:15.091747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:55:37.814180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049325+0000","last_change":"2024-03-18T23:07:13.500594+0000","last_active":"2024-03-18T23:07:15.049325+0000","last_peered":"2024-03-18T23:07:15.049325+0000","last_clean":"2024-03-18T23:07:15.049325+0000","last_became_active":"2024-03-18T23:07:13.499223+0000","last_became_peered":"2024-03-18T23:07:13.499223+0000","last_unstale":"2024-03-18T23:07:15.049325+0000","last_undegraded":"2024-03-18T23:07:15.049325+0000","last_fullsized":"2024-03-18T23:07:15.049325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:27.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104944+0000","last_change":"2024-03-18T23:07:14.062569+0000","last_active":"2024-03-18T23:07:15.104944+0000","last_peered":"2024-03-18T23:07:15.104944+0000","last_clean":"2024-03-18T23:07:15.104944+0000","last_became_active":"2024-03-18T23:07:14.062231+0000","last_became_peered":"2024-03-18T23:07:14.062231+0000","last_unstale":"2024-03-18T23:07:15.104944+0000","last_undegraded":"2024-03-18T23:07:15.104944+0000","last_fullsized":"2024-03-18T23:07:15.104944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:03.987133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863254+0000","last_change":"2024-03-18T23:07:13.502513+0000","last_active":"2024-03-18T23:07:16.863254+0000","last_peered":"2024-03-18T23:07:16.863254+0000","last_clean":"2024-03-18T23:07:16.863254+0000","last_became_active":"2024-03-18T23:07:13.502005+0000","last_became_peered":"2024-03-18T23:07:13.502005+0000","last_unstale":"2024-03-18T23:07:16.863254+0000","last_undegraded":"2024-03-18T23:07:16.863254+0000","last_fullsized":"2024-03-18T23:07:16.863254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:05:10.395264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048878+0000","last_change":"2024-03-18T23:07:14.067346+0000","last_active":"2024-03-18T23:07:15.048878+0000","last_peered":"2024-03-18T23:07:15.048878+0000","last_clean":"2024-03-18T23:07:15.048878+0000","last_became_active":"2024-03-18T23:07:14.066390+0000","last_became_peered":"2024-03-18T23:07:14.066390+0000","last_unstale":"2024-03-18T23:07:15.048878+0000","last_undegraded":"2024-03-18T23:07:15.048878+0000","last_fullsized":"2024-03-18T23:07:15.048878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:07.679527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.862857+0000","last_change":"2024-03-18T23:07:13.509887+0000","last_active":"2024-03-18T23:07:16.862857+0000","last_peered":"2024-03-18T23:07:16.862857+0000","last_clean":"2024-03-18T23:07:16.862857+0000","last_became_active":"2024-03-18T23:07:13.509631+0000","last_became_peered":"2024-03-18T23:07:13.509631+0000","last_unstale":"2024-03-18T23:07:16.862857+0000","last_undegraded":"2024-03-18T23:07:16.862857+0000","last_fullsized":"2024-03-18T23:07:16.862857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:55.782893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106061+0000","last_change":"2024-03-18T23:07:14.070297+0000","last_active":"2024-03-18T23:07:15.106061+0000","last_peered":"2024-03-18T23:07:15.106061+0000","last_clean":"2024-03-18T23:07:15.106061+0000","last_became_active":"2024-03-18T23:07:14.069696+0000","last_became_peered":"2024-03-18T23:07:14.069696+0000","last_unstale":"2024-03-18T23:07:15.106061+0000","last_undegraded":"2024-03-18T23:07:15.106061+0000","last_fullsized":"2024-03-18T23:07:15.106061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:47.448499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863053+0000","last_change":"2024-03-18T23:07:13.497436+0000","last_active":"2024-03-18T23:07:16.863053+0000","last_peered":"2024-03-18T23:07:16.863053+0000","last_clean":"2024-03-18T23:07:16.863053+0000","last_became_active":"2024-03-18T23:07:13.497090+0000","last_became_peered":"2024-03-18T23:07:13.497090+0000","last_unstale":"2024-03-18T23:07:16.863053+0000","last_undegraded":"2024-03-18T23:07:16.863053+0000","last_fullsized":"2024-03-18T23:07:16.863053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:33.496595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105823+0000","last_change":"2024-03-18T23:07:14.061399+0000","last_active":"2024-03-18T23:07:15.105823+0000","last_peered":"2024-03-18T23:07:15.105823+0000","last_clean":"2024-03-18T23:07:15.105823+0000","last_became_active":"2024-03-18T23:07:14.060753+0000","last_became_peered":"2024-03-18T23:07:14.060753+0000","last_unstale":"2024-03-18T23:07:15.105823+0000","last_undegraded":"2024-03-18T23:07:15.105823+0000","last_fullsized":"2024-03-18T23:07:15.105823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:56:21.688951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106123+0000","last_change":"2024-03-18T23:07:13.506678+0000","last_active":"2024-03-18T23:07:15.106123+0000","last_peered":"2024-03-18T23:07:15.106123+0000","last_clean":"2024-03-18T23:07:15.106123+0000","last_became_active":"2024-03-18T23:07:13.506374+0000","last_became_peered":"2024-03-18T23:07:13.506374+0000","last_unstale":"2024-03-18T23:07:15.106123+0000","last_undegraded":"2024-03-18T23:07:15.106123+0000","last_fullsized":"2024-03-18T23:07:15.106123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:03.265106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105122+0000","last_change":"2024-03-18T23:07:14.070134+0000","last_active":"2024-03-18T23:07:15.105122+0000","last_peered":"2024-03-18T23:07:15.105122+0000","last_clean":"2024-03-18T23:07:15.105122+0000","last_became_active":"2024-03-18T23:07:14.069785+0000","last_became_peered":"2024-03-18T23:07:14.069785+0000","last_unstale":"2024-03-18T23:07:15.105122+0000","last_undegraded":"2024-03-18T23:07:15.105122+0000","last_fullsized":"2024-03-18T23:07:15.105122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:17:34.975546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.865118+0000","last_change":"2024-03-18T23:07:13.502516+0000","last_active":"2024-03-18T23:07:16.865118+0000","last_peered":"2024-03-18T23:07:16.865118+0000","last_clean":"2024-03-18T23:07:16.865118+0000","last_became_active":"2024-03-18T23:07:13.502141+0000","last_became_peered":"2024-03-18T23:07:13.502141+0000","last_unstale":"2024-03-18T23:07:16.865118+0000","last_undegraded":"2024-03-18T23:07:16.865118+0000","last_fullsized":"2024-03-18T23:07:16.865118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:10.328673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846680+0000","last_change":"2024-03-18T23:07:14.070465+0000","last_active":"2024-03-18T23:07:15.846680+0000","last_peered":"2024-03-18T23:07:15.846680+0000","last_clean":"2024-03-18T23:07:15.846680+0000","last_became_active":"2024-03-18T23:07:14.070158+0000","last_became_peered":"2024-03-18T23:07:14.070158+0000","last_unstale":"2024-03-18T23:07:15.846680+0000","last_undegraded":"2024-03-18T23:07:15.846680+0000","last_fullsized":"2024-03-18T23:07:15.846680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:53.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867902+0000","last_change":"2024-03-18T23:07:13.512022+0000","last_active":"2024-03-18T23:07:17.867902+0000","last_peered":"2024-03-18T23:07:17.867902+0000","last_clean":"2024-03-18T23:07:17.867902+0000","last_became_active":"2024-03-18T23:07:13.511685+0000","last_became_peered":"2024-03-18T23:07:13.511685+0000","last_unstale":"2024-03-18T23:07:17.867902+0000","last_undegraded":"2024-03-18T23:07:17.867902+0000","last_fullsized":"2024-03-18T23:07:17.867902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:22.650105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846759+0000","last_change":"2024-03-18T23:07:14.071489+0000","last_active":"2024-03-18T23:07:15.846759+0000","last_peered":"2024-03-18T23:07:15.846759+0000","last_clean":"2024-03-18T23:07:15.846759+0000","last_became_active":"2024-03-18T23:07:14.071175+0000","last_became_peered":"2024-03-18T23:07:14.071175+0000","last_unstale":"2024-03-18T23:07:15.846759+0000","last_undegraded":"2024-03-18T23:07:15.846759+0000","last_fullsized":"2024-03-18T23:07:15.846759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:39.813451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106952+0000","last_change":"2024-03-18T23:07:13.500233+0000","last_active":"2024-03-18T23:07:15.106952+0000","last_peered":"2024-03-18T23:07:15.106952+0000","last_clean":"2024-03-18T23:07:15.106952+0000","last_became_active":"2024-03-18T23:07:13.499870+0000","last_became_peered":"2024-03-18T23:07:13.499870+0000","last_unstale":"2024-03-18T23:07:15.106952+0000","last_undegraded":"2024-03-18T23:07:15.106952+0000","last_fullsized":"2024-03-18T23:07:15.106952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:45.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.960034+0000","last_change":"2024-03-18T23:07:14.065225+0000","last_active":"2024-03-18T23:07:15.960034+0000","last_peered":"2024-03-18T23:07:15.960034+0000","last_clean":"2024-03-18T23:07:15.960034+0000","last_became_active":"2024-03-18T23:07:14.064881+0000","last_became_peered":"2024-03-18T23:07:14.064881+0000","last_unstale":"2024-03-18T23:07:15.960034+0000","last_undegraded":"2024-03-18T23:07:15.960034+0000","last_fullsized":"2024-03-18T23:07:15.960034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:31:27.921072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091784+0000","last_change":"2024-03-18T23:07:13.502565+0000","last_active":"2024-03-18T23:07:15.091784+0000","last_peered":"2024-03-18T23:07:15.091784+0000","last_clean":"2024-03-18T23:07:15.091784+0000","last_became_active":"2024-03-18T23:07:13.501777+0000","last_became_peered":"2024-03-18T23:07:13.501777+0000","last_unstale":"2024-03-18T23:07:15.091784+0000","last_undegraded":"2024-03-18T23:07:15.091784+0000","last_fullsized":"2024-03-18T23:07:15.091784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:04.482546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846766+0000","last_change":"2024-03-18T23:07:14.068691+0000","last_active":"2024-03-18T23:07:15.846766+0000","last_peered":"2024-03-18T23:07:15.846766+0000","last_clean":"2024-03-18T23:07:15.846766+0000","last_became_active":"2024-03-18T23:07:14.068268+0000","last_became_peered":"2024-03-18T23:07:14.068268+0000","last_unstale":"2024-03-18T23:07:15.846766+0000","last_undegraded":"2024-03-18T23:07:15.846766+0000","last_fullsized":"2024-03-18T23:07:15.846766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:55:56.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105857+0000","last_change":"2024-03-18T23:07:14.061079+0000","last_active":"2024-03-18T23:07:15.105857+0000","last_peered":"2024-03-18T23:07:15.105857+0000","last_clean":"2024-03-18T23:07:15.105857+0000","last_became_active":"2024-03-18T23:07:14.060379+0000","last_became_peered":"2024-03-18T23:07:14.060379+0000","last_unstale":"2024-03-18T23:07:15.105857+0000","last_undegraded":"2024-03-18T23:07:15.105857+0000","last_fullsized":"2024-03-18T23:07:15.105857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:17.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106012+0000","last_change":"2024-03-18T23:07:14.069349+0000","last_active":"2024-03-18T23:07:15.106012+0000","last_peered":"2024-03-18T23:07:15.106012+0000","last_clean":"2024-03-18T23:07:15.106012+0000","last_became_active":"2024-03-18T23:07:14.068777+0000","last_became_peered":"2024-03-18T23:07:14.068777+0000","last_unstale":"2024-03-18T23:07:15.106012+0000","last_undegraded":"2024-03-18T23:07:15.106012+0000","last_fullsized":"2024-03-18T23:07:15.106012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:32.509613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105943+0000","last_change":"2024-03-18T23:07:14.062981+0000","last_active":"2024-03-18T23:07:15.105943+0000","last_peered":"2024-03-18T23:07:15.105943+0000","last_clean":"2024-03-18T23:07:15.105943+0000","last_became_active":"2024-03-18T23:07:14.061986+0000","last_became_peered":"2024-03-18T23:07:14.061986+0000","last_unstale":"2024-03-18T23:07:15.105943+0000","last_undegraded":"2024-03-18T23:07:15.105943+0000","last_fullsized":"2024-03-18T23:07:15.105943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:46.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048763+0000","last_change":"2024-03-18T23:07:14.069200+0000","last_active":"2024-03-18T23:07:15.048763+0000","last_peered":"2024-03-18T23:07:15.048763+0000","last_clean":"2024-03-18T23:07:15.048763+0000","last_became_active":"2024-03-18T23:07:14.068893+0000","last_became_peered":"2024-03-18T23:07:14.068893+0000","last_unstale":"2024-03-18T23:07:15.048763+0000","last_undegraded":"2024-03-18T23:07:15.048763+0000","last_fullsized":"2024-03-18T23:07:15.048763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:34.488251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091858+0000","last_change":"2024-03-18T23:07:14.065429+0000","last_active":"2024-03-18T23:07:15.091858+0000","last_peered":"2024-03-18T23:07:15.091858+0000","last_clean":"2024-03-18T23:07:15.091858+0000","last_became_active":"2024-03-18T23:07:14.065083+0000","last_became_peered":"2024-03-18T23:07:14.065083+0000","last_unstale":"2024-03-18T23:07:15.091858+0000","last_undegraded":"2024-03-18T23:07:15.091858+0000","last_fullsized":"2024-03-18T23:07:15.091858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:37.317538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106805+0000","last_change":"2024-03-18T23:07:14.059155+0000","last_active":"2024-03-18T23:07:15.106805+0000","last_peered":"2024-03-18T23:07:15.106805+0000","last_clean":"2024-03-18T23:07:15.106805+0000","last_became_active":"2024-03-18T23:07:14.058800+0000","last_became_peered":"2024-03-18T23:07:14.058800+0000","last_unstale":"2024-03-18T23:07:15.106805+0000","last_undegraded":"2024-03-18T23:07:15.106805+0000","last_fullsized":"2024-03-18T23:07:15.106805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:32.994668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106991+0000","last_change":"2024-03-18T23:07:14.065351+0000","last_active":"2024-03-18T23:07:15.106991+0000","last_peered":"2024-03-18T23:07:15.106991+0000","last_clean":"2024-03-18T23:07:15.106991+0000","last_became_active":"2024-03-18T23:07:14.065012+0000","last_became_peered":"2024-03-18T23:07:14.065012+0000","last_unstale":"2024-03-18T23:07:15.106991+0000","last_undegraded":"2024-03-18T23:07:15.106991+0000","last_fullsized":"2024-03-18T23:07:15.106991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:35.452474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105108+0000","last_change":"2024-03-18T23:07:14.067179+0000","last_active":"2024-03-18T23:07:15.105108+0000","last_peered":"2024-03-18T23:07:15.105108+0000","last_clean":"2024-03-18T23:07:15.105108+0000","last_became_active":"2024-03-18T23:07:14.066899+0000","last_became_peered":"2024-03-18T23:07:14.066899+0000","last_unstale":"2024-03-18T23:07:15.105108+0000","last_undegraded":"2024-03-18T23:07:15.105108+0000","last_fullsized":"2024-03-18T23:07:15.105108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:02.180512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107527+0000","last_change":"2024-03-18T23:07:14.061089+0000","last_active":"2024-03-18T23:07:15.107527+0000","last_peered":"2024-03-18T23:07:15.107527+0000","last_clean":"2024-03-18T23:07:15.107527+0000","last_became_active":"2024-03-18T23:07:14.060787+0000","last_became_peered":"2024-03-18T23:07:14.060787+0000","last_unstale":"2024-03-18T23:07:15.107527+0000","last_undegraded":"2024-03-18T23:07:15.107527+0000","last_fullsized":"2024-03-18T23:07:15.107527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:47:52.029524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104888+0000","last_change":"2024-03-18T23:07:14.058888+0000","last_active":"2024-03-18T23:07:15.104888+0000","last_peered":"2024-03-18T23:07:15.104888+0000","last_clean":"2024-03-18T23:07:15.104888+0000","last_became_active":"2024-03-18T23:07:14.058576+0000","last_became_peered":"2024-03-18T23:07:14.058576+0000","last_unstale":"2024-03-18T23:07:15.104888+0000","last_undegraded":"2024-03-18T23:07:15.104888+0000","last_fullsized":"2024-03-18T23:07:15.104888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:27.121268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105836+0000","last_change":"2024-03-18T23:07:14.076698+0000","last_active":"2024-03-18T23:07:15.105836+0000","last_peered":"2024-03-18T23:07:15.105836+0000","last_clean":"2024-03-18T23:07:15.105836+0000","last_became_active":"2024-03-18T23:07:14.076156+0000","last_became_peered":"2024-03-18T23:07:14.076156+0000","last_unstale":"2024-03-18T23:07:15.105836+0000","last_undegraded":"2024-03-18T23:07:15.105836+0000","last_fullsized":"2024-03-18T23:07:15.105836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:56:01.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091784+0000","last_change":"2024-03-18T23:07:14.057774+0000","last_active":"2024-03-18T23:07:15.091784+0000","last_peered":"2024-03-18T23:07:15.091784+0000","last_clean":"2024-03-18T23:07:15.091784+0000","last_became_active":"2024-03-18T23:07:14.057456+0000","last_became_peered":"2024-03-18T23:07:14.057456+0000","last_unstale":"2024-03-18T23:07:15.091784+0000","last_undegraded":"2024-03-18T23:07:15.091784+0000","last_fullsized":"2024-03-18T23:07:15.091784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:33.742347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107616+0000","last_change":"2024-03-18T23:07:14.066689+0000","last_active":"2024-03-18T23:07:15.107616+0000","last_peered":"2024-03-18T23:07:15.107616+0000","last_clean":"2024-03-18T23:07:15.107616+0000","last_became_active":"2024-03-18T23:07:14.066358+0000","last_became_peered":"2024-03-18T23:07:14.066358+0000","last_unstale":"2024-03-18T23:07:15.107616+0000","last_undegraded":"2024-03-18T23:07:15.107616+0000","last_fullsized":"2024-03-18T23:07:15.107616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:29.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23793,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":112,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":40799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":946176,"data_stored":451185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":451767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972380,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":451923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234018,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1560576,"data_stored":1044244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":451185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":452797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":451879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":454147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":451469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888111,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1568768,"data_stored":1043029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:07:21.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T23:07:21.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 -- 172.21.15.132:0/972640702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa480104c90 msgr2=0x7fa480107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 --2- 172.21.15.132:0/972640702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa480104c90 0x7fa480107080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa474009900 tx=0x7fa47402bc30 comp rx=0 tx=0).stop 2024-03-18T23:07:21.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 -- 172.21.15.132:0/972640702 shutdown_connections 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 --2- 172.21.15.132:0/972640702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4801075c0 0x7fa4801099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 --2- 172.21.15.132:0/972640702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa480104c90 0x7fa480107080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 --2- 172.21.15.132:0/972640702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480102360 0x7fa480104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 -- 172.21.15.132:0/972640702 >> 172.21.15.132:0/972640702 conn(0x7fa4800fbcd0 msgr2=0x7fa4800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 -- 172.21.15.132:0/972640702 shutdown_connections 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 -- 172.21.15.132:0/972640702 wait complete. 2024-03-18T23:07:21.583 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.579+0000 7fa485ee8640 1 Processor -- start 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 -- start start 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa480102360 0x7fa480197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480104c90 0x7fa480198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010d4b0 con 0x7fa4801075c0 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010d330 con 0x7fa480102360 2024-03-18T23:07:21.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa48010d630 con 0x7fa480104c90 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51416/0 (socket says 172.21.15.132:51416) 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 -- 172.21.15.132:0/2703913796 learned_addr learned my addr 172.21.15.132:0/2703913796 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47f7fe640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa480102360 0x7fa480197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47effd640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480104c90 0x7fa480198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 -- 172.21.15.132:0/2703913796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480104c90 msgr2=0x7fa480198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480104c90 0x7fa480198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 -- 172.21.15.132:0/2703913796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa480102360 msgr2=0x7fa480197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa480102360 0x7fa480197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 -- 172.21.15.132:0/2703913796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48019f800 con 0x7fa4801075c0 2024-03-18T23:07:21.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47ffff640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa47800efd0 tx=0x7fa47800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa478003b70 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa478003d10 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48019faf0 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.580+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4780109f0 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.581+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48019ffc0 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.581+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4800fd7c0 con 0x7fa4801075c0 2024-03-18T23:07:21.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.583+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fa478010b90 con 0x7fa4801075c0 2024-03-18T23:07:21.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.583+0000 7fa47cff9640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 0x7fa4680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:21.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.583+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa4780cf1e0 con 0x7fa4801075c0 2024-03-18T23:07:21.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.583+0000 7fa47f7fe640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 0x7fa4680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:21.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.584+0000 7fa47f7fe640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 0x7fa4680a4d50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa470009bd0 tx=0x7fa470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:21.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.587+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa47809c2d0 con 0x7fa4801075c0 2024-03-18T23:07:21.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.757+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa480061c60 con 0x7fa4680a2890 2024-03-18T23:07:21.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.765+0000 7fa47cff9640 1 -- 172.21.15.132:0/2703913796 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258733 (secure 0 0 0) 0x7fa480061c60 con 0x7fa4680a2890 2024-03-18T23:07:21.770 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:21.772 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-18T23:07:21.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 msgr2=0x7fa4680a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.773 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 0x7fa4680a4d50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa470009bd0 tx=0x7fa470005f90 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 msgr2=0x7fa48019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa47800efd0 tx=0x7fa47800c6a0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 shutdown_connections 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa4680a2890 0x7fa4680a4d50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801075c0 0x7fa48019f190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa480104c90 0x7fa480198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 --2- 172.21.15.132:0/2703913796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa480102360 0x7fa480197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 >> 172.21.15.132:0/2703913796 conn(0x7fa4800fbcd0 msgr2=0x7fa480102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:21.774 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 shutdown_connections 2024-03-18T23:07:21.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:21.767+0000 7fa485ee8640 1 -- 172.21.15.132:0/2703913796 wait complete. 2024-03-18T23:07:21.778 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-18T23:07:21.044559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614395,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":152,"num_read_kb":151,"num_write":295,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":240,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370548,"kb_used_data":13028,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571276,"statfs":{"total":1151940427776,"available":1151560986624,"internally_reserved":0,"allocated":13340672,"data_stored":7206566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24027,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":114,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.557927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106381+0000","last_change":"2024-03-18T23:07:14.066593+0000","last_active":"2024-03-18T23:07:15.106381+0000","last_peered":"2024-03-18T23:07:15.106381+0000","last_clean":"2024-03-18T23:07:15.106381+0000","last_became_active":"2024-03-18T23:07:14.066064+0000","last_became_peered":"2024-03-18T23:07:14.066064+0000","last_unstale":"2024-03-18T23:07:15.106381+0000","last_undegraded":"2024-03-18T23:07:15.106381+0000","last_fullsized":"2024-03-18T23:07:15.106381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:48.122520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048742+0000","last_change":"2024-03-18T23:07:14.070484+0000","last_active":"2024-03-18T23:07:15.048742+0000","last_peered":"2024-03-18T23:07:15.048742+0000","last_clean":"2024-03-18T23:07:15.048742+0000","last_became_active":"2024-03-18T23:07:14.070129+0000","last_became_peered":"2024-03-18T23:07:14.070129+0000","last_unstale":"2024-03-18T23:07:15.048742+0000","last_undegraded":"2024-03-18T23:07:15.048742+0000","last_fullsized":"2024-03-18T23:07:15.048742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:13:44.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048820+0000","last_change":"2024-03-18T23:07:14.064923+0000","last_active":"2024-03-18T23:07:15.048820+0000","last_peered":"2024-03-18T23:07:15.048820+0000","last_clean":"2024-03-18T23:07:15.048820+0000","last_became_active":"2024-03-18T23:07:14.064513+0000","last_became_peered":"2024-03-18T23:07:14.064513+0000","last_unstale":"2024-03-18T23:07:15.048820+0000","last_undegraded":"2024-03-18T23:07:15.048820+0000","last_fullsized":"2024-03-18T23:07:15.048820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:33.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105536+0000","last_change":"2024-03-18T23:07:14.065970+0000","last_active":"2024-03-18T23:07:15.105536+0000","last_peered":"2024-03-18T23:07:15.105536+0000","last_clean":"2024-03-18T23:07:15.105536+0000","last_became_active":"2024-03-18T23:07:14.065361+0000","last_became_peered":"2024-03-18T23:07:14.065361+0000","last_unstale":"2024-03-18T23:07:15.105536+0000","last_undegraded":"2024-03-18T23:07:15.105536+0000","last_fullsized":"2024-03-18T23:07:15.105536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:54:45.552015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105736+0000","last_change":"2024-03-18T23:07:14.058801+0000","last_active":"2024-03-18T23:07:15.105736+0000","last_peered":"2024-03-18T23:07:15.105736+0000","last_clean":"2024-03-18T23:07:15.105736+0000","last_became_active":"2024-03-18T23:07:14.058481+0000","last_became_peered":"2024-03-18T23:07:14.058481+0000","last_unstale":"2024-03-18T23:07:15.105736+0000","last_undegraded":"2024-03-18T23:07:15.105736+0000","last_fullsized":"2024-03-18T23:07:15.105736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:55.420890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049487+0000","last_change":"2024-03-18T23:07:14.067320+0000","last_active":"2024-03-18T23:07:15.049487+0000","last_peered":"2024-03-18T23:07:15.049487+0000","last_clean":"2024-03-18T23:07:15.049487+0000","last_became_active":"2024-03-18T23:07:14.066987+0000","last_became_peered":"2024-03-18T23:07:14.066987+0000","last_unstale":"2024-03-18T23:07:15.049487+0000","last_undegraded":"2024-03-18T23:07:15.049487+0000","last_fullsized":"2024-03-18T23:07:15.049487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:52:33.006026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846260+0000","last_change":"2024-03-18T23:07:14.065320+0000","last_active":"2024-03-18T23:07:15.846260+0000","last_peered":"2024-03-18T23:07:15.846260+0000","last_clean":"2024-03-18T23:07:15.846260+0000","last_became_active":"2024-03-18T23:07:14.064280+0000","last_became_peered":"2024-03-18T23:07:14.064280+0000","last_unstale":"2024-03-18T23:07:15.846260+0000","last_undegraded":"2024-03-18T23:07:15.846260+0000","last_fullsized":"2024-03-18T23:07:15.846260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:07.838038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104762+0000","last_change":"2024-03-18T23:07:14.071351+0000","last_active":"2024-03-18T23:07:15.104762+0000","last_peered":"2024-03-18T23:07:15.104762+0000","last_clean":"2024-03-18T23:07:15.104762+0000","last_became_active":"2024-03-18T23:07:14.071053+0000","last_became_peered":"2024-03-18T23:07:14.071053+0000","last_unstale":"2024-03-18T23:07:15.104762+0000","last_undegraded":"2024-03-18T23:07:15.104762+0000","last_fullsized":"2024-03-18T23:07:15.104762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:28.846769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105887+0000","last_change":"2024-03-18T23:07:14.072829+0000","last_active":"2024-03-18T23:07:15.105887+0000","last_peered":"2024-03-18T23:07:15.105887+0000","last_clean":"2024-03-18T23:07:15.105887+0000","last_became_active":"2024-03-18T23:07:14.072493+0000","last_became_peered":"2024-03-18T23:07:14.072493+0000","last_unstale":"2024-03-18T23:07:15.105887+0000","last_undegraded":"2024-03-18T23:07:15.105887+0000","last_fullsized":"2024-03-18T23:07:15.105887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:58:12.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049869+0000","last_change":"2024-03-18T23:07:14.067119+0000","last_active":"2024-03-18T23:07:15.049869+0000","last_peered":"2024-03-18T23:07:15.049869+0000","last_clean":"2024-03-18T23:07:15.049869+0000","last_became_active":"2024-03-18T23:07:14.066738+0000","last_became_peered":"2024-03-18T23:07:14.066738+0000","last_unstale":"2024-03-18T23:07:15.049869+0000","last_undegraded":"2024-03-18T23:07:15.049869+0000","last_fullsized":"2024-03-18T23:07:15.049869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:55.011692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846831+0000","last_change":"2024-03-18T23:07:14.071791+0000","last_active":"2024-03-18T23:07:15.846831+0000","last_peered":"2024-03-18T23:07:15.846831+0000","last_clean":"2024-03-18T23:07:15.846831+0000","last_became_active":"2024-03-18T23:07:14.068031+0000","last_became_peered":"2024-03-18T23:07:14.068031+0000","last_unstale":"2024-03-18T23:07:15.846831+0000","last_undegraded":"2024-03-18T23:07:15.846831+0000","last_fullsized":"2024-03-18T23:07:15.846831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:42:56.142512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105994+0000","last_change":"2024-03-18T23:07:14.071712+0000","last_active":"2024-03-18T23:07:15.105994+0000","last_peered":"2024-03-18T23:07:15.105994+0000","last_clean":"2024-03-18T23:07:15.105994+0000","last_became_active":"2024-03-18T23:07:14.069595+0000","last_became_peered":"2024-03-18T23:07:14.069595+0000","last_unstale":"2024-03-18T23:07:15.105994+0000","last_undegraded":"2024-03-18T23:07:15.105994+0000","last_fullsized":"2024-03-18T23:07:15.105994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:10:21.521681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091828+0000","last_change":"2024-03-18T23:07:14.069861+0000","last_active":"2024-03-18T23:07:15.091828+0000","last_peered":"2024-03-18T23:07:15.091828+0000","last_clean":"2024-03-18T23:07:15.091828+0000","last_became_active":"2024-03-18T23:07:14.069534+0000","last_became_peered":"2024-03-18T23:07:14.069534+0000","last_unstale":"2024-03-18T23:07:15.091828+0000","last_undegraded":"2024-03-18T23:07:15.091828+0000","last_fullsized":"2024-03-18T23:07:15.091828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:14:51.360640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048494+0000","last_change":"2024-03-18T23:07:14.070755+0000","last_active":"2024-03-18T23:07:15.048494+0000","last_peered":"2024-03-18T23:07:15.048494+0000","last_clean":"2024-03-18T23:07:15.048494+0000","last_became_active":"2024-03-18T23:07:14.070126+0000","last_became_peered":"2024-03-18T23:07:14.070126+0000","last_unstale":"2024-03-18T23:07:15.048494+0000","last_undegraded":"2024-03-18T23:07:15.048494+0000","last_fullsized":"2024-03-18T23:07:15.048494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:05.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048550+0000","last_change":"2024-03-18T23:07:14.070166+0000","last_active":"2024-03-18T23:07:15.048550+0000","last_peered":"2024-03-18T23:07:15.048550+0000","last_clean":"2024-03-18T23:07:15.048550+0000","last_became_active":"2024-03-18T23:07:14.069693+0000","last_became_peered":"2024-03-18T23:07:14.069693+0000","last_unstale":"2024-03-18T23:07:15.048550+0000","last_undegraded":"2024-03-18T23:07:15.048550+0000","last_fullsized":"2024-03-18T23:07:15.048550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:55.616894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105260+0000","last_change":"2024-03-18T23:07:14.076732+0000","last_active":"2024-03-18T23:07:15.105260+0000","last_peered":"2024-03-18T23:07:15.105260+0000","last_clean":"2024-03-18T23:07:15.105260+0000","last_became_active":"2024-03-18T23:07:14.076143+0000","last_became_peered":"2024-03-18T23:07:14.076143+0000","last_unstale":"2024-03-18T23:07:15.105260+0000","last_undegraded":"2024-03-18T23:07:15.105260+0000","last_fullsized":"2024-03-18T23:07:15.105260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:46.888524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106761+0000","last_change":"2024-03-18T23:07:14.066388+0000","last_active":"2024-03-18T23:07:15.106761+0000","last_peered":"2024-03-18T23:07:15.106761+0000","last_clean":"2024-03-18T23:07:15.106761+0000","last_became_active":"2024-03-18T23:07:14.065626+0000","last_became_peered":"2024-03-18T23:07:14.065626+0000","last_unstale":"2024-03-18T23:07:15.106761+0000","last_undegraded":"2024-03-18T23:07:15.106761+0000","last_fullsized":"2024-03-18T23:07:15.106761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:08:18.200855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049523+0000","last_change":"2024-03-18T23:07:14.068730+0000","last_active":"2024-03-18T23:07:15.049523+0000","last_peered":"2024-03-18T23:07:15.049523+0000","last_clean":"2024-03-18T23:07:15.049523+0000","last_became_active":"2024-03-18T23:07:14.068420+0000","last_became_peered":"2024-03-18T23:07:14.068420+0000","last_unstale":"2024-03-18T23:07:15.049523+0000","last_undegraded":"2024-03-18T23:07:15.049523+0000","last_fullsized":"2024-03-18T23:07:15.049523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:41:34.588823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106361+0000","last_change":"2024-03-18T23:07:14.061415+0000","last_active":"2024-03-18T23:07:15.106361+0000","last_peered":"2024-03-18T23:07:15.106361+0000","last_clean":"2024-03-18T23:07:15.106361+0000","last_became_active":"2024-03-18T23:07:14.061102+0000","last_became_peered":"2024-03-18T23:07:14.061102+0000","last_unstale":"2024-03-18T23:07:15.106361+0000","last_undegraded":"2024-03-18T23:07:15.106361+0000","last_fullsized":"2024-03-18T23:07:15.106361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:06:08.056213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049174+0000","last_change":"2024-03-18T23:07:13.506404+0000","last_active":"2024-03-18T23:07:15.049174+0000","last_peered":"2024-03-18T23:07:15.049174+0000","last_clean":"2024-03-18T23:07:15.049174+0000","last_became_active":"2024-03-18T23:07:13.506091+0000","last_became_peered":"2024-03-18T23:07:13.506091+0000","last_unstale":"2024-03-18T23:07:15.049174+0000","last_undegraded":"2024-03-18T23:07:15.049174+0000","last_fullsized":"2024-03-18T23:07:15.049174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:34:26.916813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105056+0000","last_change":"2024-03-18T23:07:14.061990+0000","last_active":"2024-03-18T23:07:15.105056+0000","last_peered":"2024-03-18T23:07:15.105056+0000","last_clean":"2024-03-18T23:07:15.105056+0000","last_became_active":"2024-03-18T23:07:14.061581+0000","last_became_peered":"2024-03-18T23:07:14.061581+0000","last_unstale":"2024-03-18T23:07:15.105056+0000","last_undegraded":"2024-03-18T23:07:15.105056+0000","last_fullsized":"2024-03-18T23:07:15.105056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:37.726342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105077+0000","last_change":"2024-03-18T23:07:13.509727+0000","last_active":"2024-03-18T23:07:15.105077+0000","last_peered":"2024-03-18T23:07:15.105077+0000","last_clean":"2024-03-18T23:07:15.105077+0000","last_became_active":"2024-03-18T23:07:13.509296+0000","last_became_peered":"2024-03-18T23:07:13.509296+0000","last_unstale":"2024-03-18T23:07:15.105077+0000","last_undegraded":"2024-03-18T23:07:15.105077+0000","last_fullsized":"2024-03-18T23:07:15.105077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:15.060387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049561+0000","last_change":"2024-03-18T23:07:14.066038+0000","last_active":"2024-03-18T23:07:15.049561+0000","last_peered":"2024-03-18T23:07:15.049561+0000","last_clean":"2024-03-18T23:07:15.049561+0000","last_became_active":"2024-03-18T23:07:14.065449+0000","last_became_peered":"2024-03-18T23:07:14.065449+0000","last_unstale":"2024-03-18T23:07:15.049561+0000","last_undegraded":"2024-03-18T23:07:15.049561+0000","last_fullsized":"2024-03-18T23:07:15.049561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:53:07.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867050+0000","last_change":"2024-03-18T23:07:13.503422+0000","last_active":"2024-03-18T23:07:17.867050+0000","last_peered":"2024-03-18T23:07:17.867050+0000","last_clean":"2024-03-18T23:07:17.867050+0000","last_became_active":"2024-03-18T23:07:13.503068+0000","last_became_peered":"2024-03-18T23:07:13.503068+0000","last_unstale":"2024-03-18T23:07:17.867050+0000","last_undegraded":"2024-03-18T23:07:17.867050+0000","last_fullsized":"2024-03-18T23:07:17.867050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:35.369317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105538+0000","last_change":"2024-03-18T23:07:14.073092+0000","last_active":"2024-03-18T23:07:15.105538+0000","last_peered":"2024-03-18T23:07:15.105538+0000","last_clean":"2024-03-18T23:07:15.105538+0000","last_became_active":"2024-03-18T23:07:14.070303+0000","last_became_peered":"2024-03-18T23:07:14.070303+0000","last_unstale":"2024-03-18T23:07:15.105538+0000","last_undegraded":"2024-03-18T23:07:15.105538+0000","last_fullsized":"2024-03-18T23:07:15.105538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:09.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106947+0000","last_change":"2024-03-18T23:07:13.505420+0000","last_active":"2024-03-18T23:07:15.106947+0000","last_peered":"2024-03-18T23:07:15.106947+0000","last_clean":"2024-03-18T23:07:15.106947+0000","last_became_active":"2024-03-18T23:07:13.504995+0000","last_became_peered":"2024-03-18T23:07:13.504995+0000","last_unstale":"2024-03-18T23:07:15.106947+0000","last_undegraded":"2024-03-18T23:07:15.106947+0000","last_fullsized":"2024-03-18T23:07:15.106947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:50.988318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107146+0000","last_change":"2024-03-18T23:07:14.071254+0000","last_active":"2024-03-18T23:07:15.107146+0000","last_peered":"2024-03-18T23:07:15.107146+0000","last_clean":"2024-03-18T23:07:15.107146+0000","last_became_active":"2024-03-18T23:07:14.070767+0000","last_became_peered":"2024-03-18T23:07:14.070767+0000","last_unstale":"2024-03-18T23:07:15.107146+0000","last_undegraded":"2024-03-18T23:07:15.107146+0000","last_fullsized":"2024-03-18T23:07:15.107146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:36.639093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.866654+0000","last_change":"2024-03-18T23:07:13.505020+0000","last_active":"2024-03-18T23:07:16.866654+0000","last_peered":"2024-03-18T23:07:16.866654+0000","last_clean":"2024-03-18T23:07:16.866654+0000","last_became_active":"2024-03-18T23:07:13.504355+0000","last_became_peered":"2024-03-18T23:07:13.504355+0000","last_unstale":"2024-03-18T23:07:16.866654+0000","last_undegraded":"2024-03-18T23:07:16.866654+0000","last_fullsized":"2024-03-18T23:07:16.866654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:03.184513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105913+0000","last_change":"2024-03-18T23:07:14.063156+0000","last_active":"2024-03-18T23:07:15.105913+0000","last_peered":"2024-03-18T23:07:15.105913+0000","last_clean":"2024-03-18T23:07:15.105913+0000","last_became_active":"2024-03-18T23:07:14.062801+0000","last_became_peered":"2024-03-18T23:07:14.062801+0000","last_unstale":"2024-03-18T23:07:15.105913+0000","last_undegraded":"2024-03-18T23:07:15.105913+0000","last_fullsized":"2024-03-18T23:07:15.105913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:49:27.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867406+0000","last_change":"2024-03-18T23:07:13.501538+0000","last_active":"2024-03-18T23:07:17.867406+0000","last_peered":"2024-03-18T23:07:17.867406+0000","last_clean":"2024-03-18T23:07:17.867406+0000","last_became_active":"2024-03-18T23:07:13.501160+0000","last_became_peered":"2024-03-18T23:07:13.501160+0000","last_unstale":"2024-03-18T23:07:17.867406+0000","last_undegraded":"2024-03-18T23:07:17.867406+0000","last_fullsized":"2024-03-18T23:07:17.867406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.319636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106576+0000","last_change":"2024-03-18T23:07:14.069345+0000","last_active":"2024-03-18T23:07:15.106576+0000","last_peered":"2024-03-18T23:07:15.106576+0000","last_clean":"2024-03-18T23:07:15.106576+0000","last_became_active":"2024-03-18T23:07:14.068320+0000","last_became_peered":"2024-03-18T23:07:14.068320+0000","last_unstale":"2024-03-18T23:07:15.106576+0000","last_undegraded":"2024-03-18T23:07:15.106576+0000","last_fullsized":"2024-03-18T23:07:15.106576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:36.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091611+0000","last_change":"2024-03-18T23:07:13.507538+0000","last_active":"2024-03-18T23:07:15.091611+0000","last_peered":"2024-03-18T23:07:15.091611+0000","last_clean":"2024-03-18T23:07:15.091611+0000","last_became_active":"2024-03-18T23:07:13.506563+0000","last_became_peered":"2024-03-18T23:07:13.506563+0000","last_unstale":"2024-03-18T23:07:15.091611+0000","last_undegraded":"2024-03-18T23:07:15.091611+0000","last_fullsized":"2024-03-18T23:07:15.091611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:58.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104704+0000","last_change":"2024-03-18T23:07:14.062563+0000","last_active":"2024-03-18T23:07:15.104704+0000","last_peered":"2024-03-18T23:07:15.104704+0000","last_clean":"2024-03-18T23:07:15.104704+0000","last_became_active":"2024-03-18T23:07:14.062232+0000","last_became_peered":"2024-03-18T23:07:14.062232+0000","last_unstale":"2024-03-18T23:07:15.104704+0000","last_undegraded":"2024-03-18T23:07:15.104704+0000","last_fullsized":"2024-03-18T23:07:15.104704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:06.055127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864872+0000","last_change":"2024-03-18T23:07:13.505103+0000","last_active":"2024-03-18T23:07:16.864872+0000","last_peered":"2024-03-18T23:07:16.864872+0000","last_clean":"2024-03-18T23:07:16.864872+0000","last_became_active":"2024-03-18T23:07:13.504764+0000","last_became_peered":"2024-03-18T23:07:13.504764+0000","last_unstale":"2024-03-18T23:07:16.864872+0000","last_undegraded":"2024-03-18T23:07:16.864872+0000","last_fullsized":"2024-03-18T23:07:16.864872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:07:44.998932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049419+0000","last_change":"2024-03-18T23:07:14.066440+0000","last_active":"2024-03-18T23:07:15.049419+0000","last_peered":"2024-03-18T23:07:15.049419+0000","last_clean":"2024-03-18T23:07:15.049419+0000","last_became_active":"2024-03-18T23:07:14.066100+0000","last_became_peered":"2024-03-18T23:07:14.066100+0000","last_unstale":"2024-03-18T23:07:15.049419+0000","last_undegraded":"2024-03-18T23:07:15.049419+0000","last_fullsized":"2024-03-18T23:07:15.049419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:56:27.662447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.745939+0000","last_change":"2024-03-18T23:07:13.501952+0000","last_active":"2024-03-18T23:07:18.745939+0000","last_peered":"2024-03-18T23:07:18.745939+0000","last_clean":"2024-03-18T23:07:18.745939+0000","last_became_active":"2024-03-18T23:07:13.501628+0000","last_became_peered":"2024-03-18T23:07:13.501628+0000","last_unstale":"2024-03-18T23:07:18.745939+0000","last_undegraded":"2024-03-18T23:07:18.745939+0000","last_fullsized":"2024-03-18T23:07:18.745939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:09.046133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105985+0000","last_change":"2024-03-18T23:07:14.075917+0000","last_active":"2024-03-18T23:07:15.105985+0000","last_peered":"2024-03-18T23:07:15.105985+0000","last_clean":"2024-03-18T23:07:15.105985+0000","last_became_active":"2024-03-18T23:07:14.075522+0000","last_became_peered":"2024-03-18T23:07:14.075522+0000","last_unstale":"2024-03-18T23:07:15.105985+0000","last_undegraded":"2024-03-18T23:07:15.105985+0000","last_fullsized":"2024-03-18T23:07:15.105985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:28.108526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864211+0000","last_change":"2024-03-18T23:07:13.507271+0000","last_active":"2024-03-18T23:07:16.864211+0000","last_peered":"2024-03-18T23:07:16.864211+0000","last_clean":"2024-03-18T23:07:16.864211+0000","last_became_active":"2024-03-18T23:07:13.506771+0000","last_became_peered":"2024-03-18T23:07:13.506771+0000","last_unstale":"2024-03-18T23:07:16.864211+0000","last_undegraded":"2024-03-18T23:07:16.864211+0000","last_fullsized":"2024-03-18T23:07:16.864211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:23:08.552922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049435+0000","last_change":"2024-03-18T23:07:14.066838+0000","last_active":"2024-03-18T23:07:15.049435+0000","last_peered":"2024-03-18T23:07:15.049435+0000","last_clean":"2024-03-18T23:07:15.049435+0000","last_became_active":"2024-03-18T23:07:14.066550+0000","last_became_peered":"2024-03-18T23:07:14.066550+0000","last_unstale":"2024-03-18T23:07:15.049435+0000","last_undegraded":"2024-03-18T23:07:15.049435+0000","last_fullsized":"2024-03-18T23:07:15.049435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:59:19.048074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863902+0000","last_change":"2024-03-18T23:07:13.511861+0000","last_active":"2024-03-18T23:07:16.863902+0000","last_peered":"2024-03-18T23:07:16.863902+0000","last_clean":"2024-03-18T23:07:16.863902+0000","last_became_active":"2024-03-18T23:07:13.511458+0000","last_became_peered":"2024-03-18T23:07:13.511458+0000","last_unstale":"2024-03-18T23:07:16.863902+0000","last_undegraded":"2024-03-18T23:07:16.863902+0000","last_fullsized":"2024-03-18T23:07:16.863902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:03.301572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105849+0000","last_change":"2024-03-18T23:07:14.073392+0000","last_active":"2024-03-18T23:07:15.105849+0000","last_peered":"2024-03-18T23:07:15.105849+0000","last_clean":"2024-03-18T23:07:15.105849+0000","last_became_active":"2024-03-18T23:07:14.071962+0000","last_became_peered":"2024-03-18T23:07:14.071962+0000","last_unstale":"2024-03-18T23:07:15.105849+0000","last_undegraded":"2024-03-18T23:07:15.105849+0000","last_fullsized":"2024-03-18T23:07:15.105849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:55:01.175354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:19.749139+0000","last_change":"2024-03-18T23:07:13.505399+0000","last_active":"2024-03-18T23:07:19.749139+0000","last_peered":"2024-03-18T23:07:19.749139+0000","last_clean":"2024-03-18T23:07:19.749139+0000","last_became_active":"2024-03-18T23:07:13.505034+0000","last_became_peered":"2024-03-18T23:07:13.505034+0000","last_unstale":"2024-03-18T23:07:19.749139+0000","last_undegraded":"2024-03-18T23:07:19.749139+0000","last_fullsized":"2024-03-18T23:07:19.749139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:52.665189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106750+0000","last_change":"2024-03-18T23:07:14.059215+0000","last_active":"2024-03-18T23:07:15.106750+0000","last_peered":"2024-03-18T23:07:15.106750+0000","last_clean":"2024-03-18T23:07:15.106750+0000","last_became_active":"2024-03-18T23:07:14.058904+0000","last_became_peered":"2024-03-18T23:07:14.058904+0000","last_unstale":"2024-03-18T23:07:15.106750+0000","last_undegraded":"2024-03-18T23:07:15.106750+0000","last_fullsized":"2024-03-18T23:07:15.106750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:28:38.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.869100+0000","last_change":"2024-03-18T23:07:13.503163+0000","last_active":"2024-03-18T23:07:17.869100+0000","last_peered":"2024-03-18T23:07:17.869100+0000","last_clean":"2024-03-18T23:07:17.869100+0000","last_became_active":"2024-03-18T23:07:13.502816+0000","last_became_peered":"2024-03-18T23:07:13.502816+0000","last_unstale":"2024-03-18T23:07:17.869100+0000","last_undegraded":"2024-03-18T23:07:17.869100+0000","last_fullsized":"2024-03-18T23:07:17.869100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:38:23.950178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091679+0000","last_change":"2024-03-18T23:07:14.057680+0000","last_active":"2024-03-18T23:07:15.091679+0000","last_peered":"2024-03-18T23:07:15.091679+0000","last_clean":"2024-03-18T23:07:15.091679+0000","last_became_active":"2024-03-18T23:07:14.057344+0000","last_became_peered":"2024-03-18T23:07:14.057344+0000","last_unstale":"2024-03-18T23:07:15.091679+0000","last_undegraded":"2024-03-18T23:07:15.091679+0000","last_fullsized":"2024-03-18T23:07:15.091679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:40.616782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.864726+0000","last_change":"2024-03-18T23:07:13.500994+0000","last_active":"2024-03-18T23:07:16.864726+0000","last_peered":"2024-03-18T23:07:16.864726+0000","last_clean":"2024-03-18T23:07:16.864726+0000","last_became_active":"2024-03-18T23:07:13.500678+0000","last_became_peered":"2024-03-18T23:07:13.500678+0000","last_unstale":"2024-03-18T23:07:16.864726+0000","last_undegraded":"2024-03-18T23:07:16.864726+0000","last_fullsized":"2024-03-18T23:07:16.864726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:41:30.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846633+0000","last_change":"2024-03-18T23:07:14.073945+0000","last_active":"2024-03-18T23:07:15.846633+0000","last_peered":"2024-03-18T23:07:15.846633+0000","last_clean":"2024-03-18T23:07:15.846633+0000","last_became_active":"2024-03-18T23:07:14.073561+0000","last_became_peered":"2024-03-18T23:07:14.073561+0000","last_unstale":"2024-03-18T23:07:15.846633+0000","last_undegraded":"2024-03-18T23:07:15.846633+0000","last_fullsized":"2024-03-18T23:07:15.846633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:33:26.514969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105115+0000","last_change":"2024-03-18T23:07:13.502121+0000","last_active":"2024-03-18T23:07:15.105115+0000","last_peered":"2024-03-18T23:07:15.105115+0000","last_clean":"2024-03-18T23:07:15.105115+0000","last_became_active":"2024-03-18T23:07:13.501796+0000","last_became_peered":"2024-03-18T23:07:13.501796+0000","last_unstale":"2024-03-18T23:07:15.105115+0000","last_undegraded":"2024-03-18T23:07:15.105115+0000","last_fullsized":"2024-03-18T23:07:15.105115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:02.348416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846134+0000","last_change":"2024-03-18T23:07:14.074062+0000","last_active":"2024-03-18T23:07:15.846134+0000","last_peered":"2024-03-18T23:07:15.846134+0000","last_clean":"2024-03-18T23:07:15.846134+0000","last_became_active":"2024-03-18T23:07:14.073657+0000","last_became_peered":"2024-03-18T23:07:14.073657+0000","last_unstale":"2024-03-18T23:07:15.846134+0000","last_undegraded":"2024-03-18T23:07:15.846134+0000","last_fullsized":"2024-03-18T23:07:15.846134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:28.103535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.868925+0000","last_change":"2024-03-18T23:07:13.509334+0000","last_active":"2024-03-18T23:07:17.868925+0000","last_peered":"2024-03-18T23:07:17.868925+0000","last_clean":"2024-03-18T23:07:17.868925+0000","last_became_active":"2024-03-18T23:07:13.509036+0000","last_became_peered":"2024-03-18T23:07:13.509036+0000","last_unstale":"2024-03-18T23:07:17.868925+0000","last_undegraded":"2024-03-18T23:07:17.868925+0000","last_fullsized":"2024-03-18T23:07:17.868925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:14:27.935984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107718+0000","last_change":"2024-03-18T23:07:14.068751+0000","last_active":"2024-03-18T23:07:15.107718+0000","last_peered":"2024-03-18T23:07:15.107718+0000","last_clean":"2024-03-18T23:07:15.107718+0000","last_became_active":"2024-03-18T23:07:14.068417+0000","last_became_peered":"2024-03-18T23:07:14.068417+0000","last_unstale":"2024-03-18T23:07:15.107718+0000","last_undegraded":"2024-03-18T23:07:15.107718+0000","last_fullsized":"2024-03-18T23:07:15.107718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:26:31.485179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:18.745023+0000","last_change":"2024-03-18T23:07:13.499616+0000","last_active":"2024-03-18T23:07:18.745023+0000","last_peered":"2024-03-18T23:07:18.745023+0000","last_clean":"2024-03-18T23:07:18.745023+0000","last_became_active":"2024-03-18T23:07:13.499129+0000","last_became_peered":"2024-03-18T23:07:13.499129+0000","last_unstale":"2024-03-18T23:07:18.745023+0000","last_undegraded":"2024-03-18T23:07:18.745023+0000","last_fullsized":"2024-03-18T23:07:18.745023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:28.699813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106374+0000","last_change":"2024-03-18T23:07:14.066305+0000","last_active":"2024-03-18T23:07:15.106374+0000","last_peered":"2024-03-18T23:07:15.106374+0000","last_clean":"2024-03-18T23:07:15.106374+0000","last_became_active":"2024-03-18T23:07:14.065952+0000","last_became_peered":"2024-03-18T23:07:14.065952+0000","last_unstale":"2024-03-18T23:07:15.106374+0000","last_undegraded":"2024-03-18T23:07:15.106374+0000","last_fullsized":"2024-03-18T23:07:15.106374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:06:00.718689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049291+0000","last_change":"2024-03-18T23:07:13.508496+0000","last_active":"2024-03-18T23:07:15.049291+0000","last_peered":"2024-03-18T23:07:15.049291+0000","last_clean":"2024-03-18T23:07:15.049291+0000","last_became_active":"2024-03-18T23:07:13.508041+0000","last_became_peered":"2024-03-18T23:07:13.508041+0000","last_unstale":"2024-03-18T23:07:15.049291+0000","last_undegraded":"2024-03-18T23:07:15.049291+0000","last_fullsized":"2024-03-18T23:07:15.049291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:22.989161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.102627+0000","last_change":"2024-03-18T23:07:14.072979+0000","last_active":"2024-03-18T23:07:15.102627+0000","last_peered":"2024-03-18T23:07:15.102627+0000","last_clean":"2024-03-18T23:07:15.102627+0000","last_became_active":"2024-03-18T23:07:14.072663+0000","last_became_peered":"2024-03-18T23:07:14.072663+0000","last_unstale":"2024-03-18T23:07:15.102627+0000","last_undegraded":"2024-03-18T23:07:15.102627+0000","last_fullsized":"2024-03-18T23:07:15.102627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:19.916681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.959996+0000","last_change":"2024-03-18T23:07:13.506789+0000","last_active":"2024-03-18T23:07:15.959996+0000","last_peered":"2024-03-18T23:07:15.959996+0000","last_clean":"2024-03-18T23:07:15.959996+0000","last_became_active":"2024-03-18T23:07:13.506412+0000","last_became_peered":"2024-03-18T23:07:13.506412+0000","last_unstale":"2024-03-18T23:07:15.959996+0000","last_undegraded":"2024-03-18T23:07:15.959996+0000","last_fullsized":"2024-03-18T23:07:15.959996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:37.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049560+0000","last_change":"2024-03-18T23:07:14.071419+0000","last_active":"2024-03-18T23:07:15.049560+0000","last_peered":"2024-03-18T23:07:15.049560+0000","last_clean":"2024-03-18T23:07:15.049560+0000","last_became_active":"2024-03-18T23:07:14.071071+0000","last_became_peered":"2024-03-18T23:07:14.071071+0000","last_unstale":"2024-03-18T23:07:15.049560+0000","last_undegraded":"2024-03-18T23:07:15.049560+0000","last_fullsized":"2024-03-18T23:07:15.049560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:25.657312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107514+0000","last_change":"2024-03-18T23:07:14.065418+0000","last_active":"2024-03-18T23:07:15.107514+0000","last_peered":"2024-03-18T23:07:15.107514+0000","last_clean":"2024-03-18T23:07:15.107514+0000","last_became_active":"2024-03-18T23:07:14.065057+0000","last_became_peered":"2024-03-18T23:07:14.065057+0000","last_unstale":"2024-03-18T23:07:15.107514+0000","last_undegraded":"2024-03-18T23:07:15.107514+0000","last_fullsized":"2024-03-18T23:07:15.107514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:23:29.531861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091513+0000","last_change":"2024-03-18T23:07:13.510913+0000","last_active":"2024-03-18T23:07:15.091513+0000","last_peered":"2024-03-18T23:07:15.091513+0000","last_clean":"2024-03-18T23:07:15.091513+0000","last_became_active":"2024-03-18T23:07:13.510563+0000","last_became_peered":"2024-03-18T23:07:13.510563+0000","last_unstale":"2024-03-18T23:07:15.091513+0000","last_undegraded":"2024-03-18T23:07:15.091513+0000","last_fullsized":"2024-03-18T23:07:15.091513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:15.798492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105244+0000","last_change":"2024-03-18T23:07:14.065203+0000","last_active":"2024-03-18T23:07:15.105244+0000","last_peered":"2024-03-18T23:07:15.105244+0000","last_clean":"2024-03-18T23:07:15.105244+0000","last_became_active":"2024-03-18T23:07:14.064924+0000","last_became_peered":"2024-03-18T23:07:14.064924+0000","last_unstale":"2024-03-18T23:07:15.105244+0000","last_undegraded":"2024-03-18T23:07:15.105244+0000","last_fullsized":"2024-03-18T23:07:15.105244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:32:11.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863077+0000","last_change":"2024-03-18T23:07:13.495244+0000","last_active":"2024-03-18T23:07:16.863077+0000","last_peered":"2024-03-18T23:07:16.863077+0000","last_clean":"2024-03-18T23:07:16.863077+0000","last_became_active":"2024-03-18T23:07:13.494879+0000","last_became_peered":"2024-03-18T23:07:13.494879+0000","last_unstale":"2024-03-18T23:07:16.863077+0000","last_undegraded":"2024-03-18T23:07:16.863077+0000","last_fullsized":"2024-03-18T23:07:16.863077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:14:51.684956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846102+0000","last_change":"2024-03-18T23:06:03.931253+0000","last_active":"2024-03-18T23:07:15.846102+0000","last_peered":"2024-03-18T23:07:15.846102+0000","last_clean":"2024-03-18T23:07:15.846102+0000","last_became_active":"2024-03-18T23:06:03.930687+0000","last_became_peered":"2024-03-18T23:06:03.930687+0000","last_unstale":"2024-03-18T23:07:15.846102+0000","last_undegraded":"2024-03-18T23:07:15.846102+0000","last_fullsized":"2024-03-18T23:07:15.846102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_clean_scrub_stamp":"2024-03-18T23:02:45.345904+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:08:47.199331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091856+0000","last_change":"2024-03-18T23:07:14.065457+0000","last_active":"2024-03-18T23:07:15.091856+0000","last_peered":"2024-03-18T23:07:15.091856+0000","last_clean":"2024-03-18T23:07:15.091856+0000","last_became_active":"2024-03-18T23:07:14.065117+0000","last_became_peered":"2024-03-18T23:07:14.065117+0000","last_unstale":"2024-03-18T23:07:15.091856+0000","last_undegraded":"2024-03-18T23:07:15.091856+0000","last_fullsized":"2024-03-18T23:07:15.091856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:43.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.868349+0000","last_change":"2024-03-18T23:07:13.501074+0000","last_active":"2024-03-18T23:07:17.868349+0000","last_peered":"2024-03-18T23:07:17.868349+0000","last_clean":"2024-03-18T23:07:17.868349+0000","last_became_active":"2024-03-18T23:07:13.500549+0000","last_became_peered":"2024-03-18T23:07:13.500549+0000","last_unstale":"2024-03-18T23:07:17.868349+0000","last_undegraded":"2024-03-18T23:07:17.868349+0000","last_fullsized":"2024-03-18T23:07:17.868349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:53:04.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863994+0000","last_change":"2024-03-18T23:07:13.510215+0000","last_active":"2024-03-18T23:07:16.863994+0000","last_peered":"2024-03-18T23:07:16.863994+0000","last_clean":"2024-03-18T23:07:16.863994+0000","last_became_active":"2024-03-18T23:07:13.509864+0000","last_became_peered":"2024-03-18T23:07:13.509864+0000","last_unstale":"2024-03-18T23:07:16.863994+0000","last_undegraded":"2024-03-18T23:07:16.863994+0000","last_fullsized":"2024-03-18T23:07:16.863994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:00:35.421397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091747+0000","last_change":"2024-03-18T23:07:14.058971+0000","last_active":"2024-03-18T23:07:15.091747+0000","last_peered":"2024-03-18T23:07:15.091747+0000","last_clean":"2024-03-18T23:07:15.091747+0000","last_became_active":"2024-03-18T23:07:14.058619+0000","last_became_peered":"2024-03-18T23:07:14.058619+0000","last_unstale":"2024-03-18T23:07:15.091747+0000","last_undegraded":"2024-03-18T23:07:15.091747+0000","last_fullsized":"2024-03-18T23:07:15.091747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:55:37.814180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.049325+0000","last_change":"2024-03-18T23:07:13.500594+0000","last_active":"2024-03-18T23:07:15.049325+0000","last_peered":"2024-03-18T23:07:15.049325+0000","last_clean":"2024-03-18T23:07:15.049325+0000","last_became_active":"2024-03-18T23:07:13.499223+0000","last_became_peered":"2024-03-18T23:07:13.499223+0000","last_unstale":"2024-03-18T23:07:15.049325+0000","last_undegraded":"2024-03-18T23:07:15.049325+0000","last_fullsized":"2024-03-18T23:07:15.049325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:27.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104944+0000","last_change":"2024-03-18T23:07:14.062569+0000","last_active":"2024-03-18T23:07:15.104944+0000","last_peered":"2024-03-18T23:07:15.104944+0000","last_clean":"2024-03-18T23:07:15.104944+0000","last_became_active":"2024-03-18T23:07:14.062231+0000","last_became_peered":"2024-03-18T23:07:14.062231+0000","last_unstale":"2024-03-18T23:07:15.104944+0000","last_undegraded":"2024-03-18T23:07:15.104944+0000","last_fullsized":"2024-03-18T23:07:15.104944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:03.987133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863254+0000","last_change":"2024-03-18T23:07:13.502513+0000","last_active":"2024-03-18T23:07:16.863254+0000","last_peered":"2024-03-18T23:07:16.863254+0000","last_clean":"2024-03-18T23:07:16.863254+0000","last_became_active":"2024-03-18T23:07:13.502005+0000","last_became_peered":"2024-03-18T23:07:13.502005+0000","last_unstale":"2024-03-18T23:07:16.863254+0000","last_undegraded":"2024-03-18T23:07:16.863254+0000","last_fullsized":"2024-03-18T23:07:16.863254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:05:10.395264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048878+0000","last_change":"2024-03-18T23:07:14.067346+0000","last_active":"2024-03-18T23:07:15.048878+0000","last_peered":"2024-03-18T23:07:15.048878+0000","last_clean":"2024-03-18T23:07:15.048878+0000","last_became_active":"2024-03-18T23:07:14.066390+0000","last_became_peered":"2024-03-18T23:07:14.066390+0000","last_unstale":"2024-03-18T23:07:15.048878+0000","last_undegraded":"2024-03-18T23:07:15.048878+0000","last_fullsized":"2024-03-18T23:07:15.048878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:07.679527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.862857+0000","last_change":"2024-03-18T23:07:13.509887+0000","last_active":"2024-03-18T23:07:16.862857+0000","last_peered":"2024-03-18T23:07:16.862857+0000","last_clean":"2024-03-18T23:07:16.862857+0000","last_became_active":"2024-03-18T23:07:13.509631+0000","last_became_peered":"2024-03-18T23:07:13.509631+0000","last_unstale":"2024-03-18T23:07:16.862857+0000","last_undegraded":"2024-03-18T23:07:16.862857+0000","last_fullsized":"2024-03-18T23:07:16.862857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:55.782893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106061+0000","last_change":"2024-03-18T23:07:14.070297+0000","last_active":"2024-03-18T23:07:15.106061+0000","last_peered":"2024-03-18T23:07:15.106061+0000","last_clean":"2024-03-18T23:07:15.106061+0000","last_became_active":"2024-03-18T23:07:14.069696+0000","last_became_peered":"2024-03-18T23:07:14.069696+0000","last_unstale":"2024-03-18T23:07:15.106061+0000","last_undegraded":"2024-03-18T23:07:15.106061+0000","last_fullsized":"2024-03-18T23:07:15.106061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:47.448499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.863053+0000","last_change":"2024-03-18T23:07:13.497436+0000","last_active":"2024-03-18T23:07:16.863053+0000","last_peered":"2024-03-18T23:07:16.863053+0000","last_clean":"2024-03-18T23:07:16.863053+0000","last_became_active":"2024-03-18T23:07:13.497090+0000","last_became_peered":"2024-03-18T23:07:13.497090+0000","last_unstale":"2024-03-18T23:07:16.863053+0000","last_undegraded":"2024-03-18T23:07:16.863053+0000","last_fullsized":"2024-03-18T23:07:16.863053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:33.496595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105823+0000","last_change":"2024-03-18T23:07:14.061399+0000","last_active":"2024-03-18T23:07:15.105823+0000","last_peered":"2024-03-18T23:07:15.105823+0000","last_clean":"2024-03-18T23:07:15.105823+0000","last_became_active":"2024-03-18T23:07:14.060753+0000","last_became_peered":"2024-03-18T23:07:14.060753+0000","last_unstale":"2024-03-18T23:07:15.105823+0000","last_undegraded":"2024-03-18T23:07:15.105823+0000","last_fullsized":"2024-03-18T23:07:15.105823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:56:21.688951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106123+0000","last_change":"2024-03-18T23:07:13.506678+0000","last_active":"2024-03-18T23:07:15.106123+0000","last_peered":"2024-03-18T23:07:15.106123+0000","last_clean":"2024-03-18T23:07:15.106123+0000","last_became_active":"2024-03-18T23:07:13.506374+0000","last_became_peered":"2024-03-18T23:07:13.506374+0000","last_unstale":"2024-03-18T23:07:15.106123+0000","last_undegraded":"2024-03-18T23:07:15.106123+0000","last_fullsized":"2024-03-18T23:07:15.106123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:03.265106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105122+0000","last_change":"2024-03-18T23:07:14.070134+0000","last_active":"2024-03-18T23:07:15.105122+0000","last_peered":"2024-03-18T23:07:15.105122+0000","last_clean":"2024-03-18T23:07:15.105122+0000","last_became_active":"2024-03-18T23:07:14.069785+0000","last_became_peered":"2024-03-18T23:07:14.069785+0000","last_unstale":"2024-03-18T23:07:15.105122+0000","last_undegraded":"2024-03-18T23:07:15.105122+0000","last_fullsized":"2024-03-18T23:07:15.105122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:17:34.975546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:16.865118+0000","last_change":"2024-03-18T23:07:13.502516+0000","last_active":"2024-03-18T23:07:16.865118+0000","last_peered":"2024-03-18T23:07:16.865118+0000","last_clean":"2024-03-18T23:07:16.865118+0000","last_became_active":"2024-03-18T23:07:13.502141+0000","last_became_peered":"2024-03-18T23:07:13.502141+0000","last_unstale":"2024-03-18T23:07:16.865118+0000","last_undegraded":"2024-03-18T23:07:16.865118+0000","last_fullsized":"2024-03-18T23:07:16.865118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:10.328673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846680+0000","last_change":"2024-03-18T23:07:14.070465+0000","last_active":"2024-03-18T23:07:15.846680+0000","last_peered":"2024-03-18T23:07:15.846680+0000","last_clean":"2024-03-18T23:07:15.846680+0000","last_became_active":"2024-03-18T23:07:14.070158+0000","last_became_peered":"2024-03-18T23:07:14.070158+0000","last_unstale":"2024-03-18T23:07:15.846680+0000","last_undegraded":"2024-03-18T23:07:15.846680+0000","last_fullsized":"2024-03-18T23:07:15.846680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:53.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:17.867902+0000","last_change":"2024-03-18T23:07:13.512022+0000","last_active":"2024-03-18T23:07:17.867902+0000","last_peered":"2024-03-18T23:07:17.867902+0000","last_clean":"2024-03-18T23:07:17.867902+0000","last_became_active":"2024-03-18T23:07:13.511685+0000","last_became_peered":"2024-03-18T23:07:13.511685+0000","last_unstale":"2024-03-18T23:07:17.867902+0000","last_undegraded":"2024-03-18T23:07:17.867902+0000","last_fullsized":"2024-03-18T23:07:17.867902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:22.650105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846759+0000","last_change":"2024-03-18T23:07:14.071489+0000","last_active":"2024-03-18T23:07:15.846759+0000","last_peered":"2024-03-18T23:07:15.846759+0000","last_clean":"2024-03-18T23:07:15.846759+0000","last_became_active":"2024-03-18T23:07:14.071175+0000","last_became_peered":"2024-03-18T23:07:14.071175+0000","last_unstale":"2024-03-18T23:07:15.846759+0000","last_undegraded":"2024-03-18T23:07:15.846759+0000","last_fullsized":"2024-03-18T23:07:15.846759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:39.813451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106952+0000","last_change":"2024-03-18T23:07:13.500233+0000","last_active":"2024-03-18T23:07:15.106952+0000","last_peered":"2024-03-18T23:07:15.106952+0000","last_clean":"2024-03-18T23:07:15.106952+0000","last_became_active":"2024-03-18T23:07:13.499870+0000","last_became_peered":"2024-03-18T23:07:13.499870+0000","last_unstale":"2024-03-18T23:07:15.106952+0000","last_undegraded":"2024-03-18T23:07:15.106952+0000","last_fullsized":"2024-03-18T23:07:15.106952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:45.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.960034+0000","last_change":"2024-03-18T23:07:14.065225+0000","last_active":"2024-03-18T23:07:15.960034+0000","last_peered":"2024-03-18T23:07:15.960034+0000","last_clean":"2024-03-18T23:07:15.960034+0000","last_became_active":"2024-03-18T23:07:14.064881+0000","last_became_peered":"2024-03-18T23:07:14.064881+0000","last_unstale":"2024-03-18T23:07:15.960034+0000","last_undegraded":"2024-03-18T23:07:15.960034+0000","last_fullsized":"2024-03-18T23:07:15.960034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:31:27.921072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091784+0000","last_change":"2024-03-18T23:07:13.502565+0000","last_active":"2024-03-18T23:07:15.091784+0000","last_peered":"2024-03-18T23:07:15.091784+0000","last_clean":"2024-03-18T23:07:15.091784+0000","last_became_active":"2024-03-18T23:07:13.501777+0000","last_became_peered":"2024-03-18T23:07:13.501777+0000","last_unstale":"2024-03-18T23:07:15.091784+0000","last_undegraded":"2024-03-18T23:07:15.091784+0000","last_fullsized":"2024-03-18T23:07:15.091784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:04.482546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.846766+0000","last_change":"2024-03-18T23:07:14.068691+0000","last_active":"2024-03-18T23:07:15.846766+0000","last_peered":"2024-03-18T23:07:15.846766+0000","last_clean":"2024-03-18T23:07:15.846766+0000","last_became_active":"2024-03-18T23:07:14.068268+0000","last_became_peered":"2024-03-18T23:07:14.068268+0000","last_unstale":"2024-03-18T23:07:15.846766+0000","last_undegraded":"2024-03-18T23:07:15.846766+0000","last_fullsized":"2024-03-18T23:07:15.846766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:55:56.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105857+0000","last_change":"2024-03-18T23:07:14.061079+0000","last_active":"2024-03-18T23:07:15.105857+0000","last_peered":"2024-03-18T23:07:15.105857+0000","last_clean":"2024-03-18T23:07:15.105857+0000","last_became_active":"2024-03-18T23:07:14.060379+0000","last_became_peered":"2024-03-18T23:07:14.060379+0000","last_unstale":"2024-03-18T23:07:15.105857+0000","last_undegraded":"2024-03-18T23:07:15.105857+0000","last_fullsized":"2024-03-18T23:07:15.105857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:17.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106012+0000","last_change":"2024-03-18T23:07:14.069349+0000","last_active":"2024-03-18T23:07:15.106012+0000","last_peered":"2024-03-18T23:07:15.106012+0000","last_clean":"2024-03-18T23:07:15.106012+0000","last_became_active":"2024-03-18T23:07:14.068777+0000","last_became_peered":"2024-03-18T23:07:14.068777+0000","last_unstale":"2024-03-18T23:07:15.106012+0000","last_undegraded":"2024-03-18T23:07:15.106012+0000","last_fullsized":"2024-03-18T23:07:15.106012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:32.509613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105943+0000","last_change":"2024-03-18T23:07:14.062981+0000","last_active":"2024-03-18T23:07:15.105943+0000","last_peered":"2024-03-18T23:07:15.105943+0000","last_clean":"2024-03-18T23:07:15.105943+0000","last_became_active":"2024-03-18T23:07:14.061986+0000","last_became_peered":"2024-03-18T23:07:14.061986+0000","last_unstale":"2024-03-18T23:07:15.105943+0000","last_undegraded":"2024-03-18T23:07:15.105943+0000","last_fullsized":"2024-03-18T23:07:15.105943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:46.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.048763+0000","last_change":"2024-03-18T23:07:14.069200+0000","last_active":"2024-03-18T23:07:15.048763+0000","last_peered":"2024-03-18T23:07:15.048763+0000","last_clean":"2024-03-18T23:07:15.048763+0000","last_became_active":"2024-03-18T23:07:14.068893+0000","last_became_peered":"2024-03-18T23:07:14.068893+0000","last_unstale":"2024-03-18T23:07:15.048763+0000","last_undegraded":"2024-03-18T23:07:15.048763+0000","last_fullsized":"2024-03-18T23:07:15.048763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:34.488251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091858+0000","last_change":"2024-03-18T23:07:14.065429+0000","last_active":"2024-03-18T23:07:15.091858+0000","last_peered":"2024-03-18T23:07:15.091858+0000","last_clean":"2024-03-18T23:07:15.091858+0000","last_became_active":"2024-03-18T23:07:14.065083+0000","last_became_peered":"2024-03-18T23:07:14.065083+0000","last_unstale":"2024-03-18T23:07:15.091858+0000","last_undegraded":"2024-03-18T23:07:15.091858+0000","last_fullsized":"2024-03-18T23:07:15.091858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:37.317538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106805+0000","last_change":"2024-03-18T23:07:14.059155+0000","last_active":"2024-03-18T23:07:15.106805+0000","last_peered":"2024-03-18T23:07:15.106805+0000","last_clean":"2024-03-18T23:07:15.106805+0000","last_became_active":"2024-03-18T23:07:14.058800+0000","last_became_peered":"2024-03-18T23:07:14.058800+0000","last_unstale":"2024-03-18T23:07:15.106805+0000","last_undegraded":"2024-03-18T23:07:15.106805+0000","last_fullsized":"2024-03-18T23:07:15.106805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:32.994668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.106991+0000","last_change":"2024-03-18T23:07:14.065351+0000","last_active":"2024-03-18T23:07:15.106991+0000","last_peered":"2024-03-18T23:07:15.106991+0000","last_clean":"2024-03-18T23:07:15.106991+0000","last_became_active":"2024-03-18T23:07:14.065012+0000","last_became_peered":"2024-03-18T23:07:14.065012+0000","last_unstale":"2024-03-18T23:07:15.106991+0000","last_undegraded":"2024-03-18T23:07:15.106991+0000","last_fullsized":"2024-03-18T23:07:15.106991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:35.452474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105108+0000","last_change":"2024-03-18T23:07:14.067179+0000","last_active":"2024-03-18T23:07:15.105108+0000","last_peered":"2024-03-18T23:07:15.105108+0000","last_clean":"2024-03-18T23:07:15.105108+0000","last_became_active":"2024-03-18T23:07:14.066899+0000","last_became_peered":"2024-03-18T23:07:14.066899+0000","last_unstale":"2024-03-18T23:07:15.105108+0000","last_undegraded":"2024-03-18T23:07:15.105108+0000","last_fullsized":"2024-03-18T23:07:15.105108+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:02.180512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107527+0000","last_change":"2024-03-18T23:07:14.061089+0000","last_active":"2024-03-18T23:07:15.107527+0000","last_peered":"2024-03-18T23:07:15.107527+0000","last_clean":"2024-03-18T23:07:15.107527+0000","last_became_active":"2024-03-18T23:07:14.060787+0000","last_became_peered":"2024-03-18T23:07:14.060787+0000","last_unstale":"2024-03-18T23:07:15.107527+0000","last_undegraded":"2024-03-18T23:07:15.107527+0000","last_fullsized":"2024-03-18T23:07:15.107527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:47:52.029524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.104888+0000","last_change":"2024-03-18T23:07:14.058888+0000","last_active":"2024-03-18T23:07:15.104888+0000","last_peered":"2024-03-18T23:07:15.104888+0000","last_clean":"2024-03-18T23:07:15.104888+0000","last_became_active":"2024-03-18T23:07:14.058576+0000","last_became_peered":"2024-03-18T23:07:14.058576+0000","last_unstale":"2024-03-18T23:07:15.104888+0000","last_undegraded":"2024-03-18T23:07:15.104888+0000","last_fullsized":"2024-03-18T23:07:15.104888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:27.121268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.105836+0000","last_change":"2024-03-18T23:07:14.076698+0000","last_active":"2024-03-18T23:07:15.105836+0000","last_peered":"2024-03-18T23:07:15.105836+0000","last_clean":"2024-03-18T23:07:15.105836+0000","last_became_active":"2024-03-18T23:07:14.076156+0000","last_became_peered":"2024-03-18T23:07:14.076156+0000","last_unstale":"2024-03-18T23:07:15.105836+0000","last_undegraded":"2024-03-18T23:07:15.105836+0000","last_fullsized":"2024-03-18T23:07:15.105836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:56:01.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.091784+0000","last_change":"2024-03-18T23:07:14.057774+0000","last_active":"2024-03-18T23:07:15.091784+0000","last_peered":"2024-03-18T23:07:15.091784+0000","last_clean":"2024-03-18T23:07:15.091784+0000","last_became_active":"2024-03-18T23:07:14.057456+0000","last_became_peered":"2024-03-18T23:07:14.057456+0000","last_unstale":"2024-03-18T23:07:15.091784+0000","last_undegraded":"2024-03-18T23:07:15.091784+0000","last_fullsized":"2024-03-18T23:07:15.091784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:33.742347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T23:07:15.107616+0000","last_change":"2024-03-18T23:07:14.066689+0000","last_active":"2024-03-18T23:07:15.107616+0000","last_peered":"2024-03-18T23:07:15.107616+0000","last_clean":"2024-03-18T23:07:15.107616+0000","last_became_active":"2024-03-18T23:07:14.066358+0000","last_became_peered":"2024-03-18T23:07:14.066358+0000","last_unstale":"2024-03-18T23:07:15.107616+0000","last_undegraded":"2024-03-18T23:07:15.107616+0000","last_fullsized":"2024-03-18T23:07:15.107616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:29.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23793,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":112,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":40799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":946176,"data_stored":451185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1051734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":451767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972380,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":451923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234018,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1560576,"data_stored":1044244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":451185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":958464,"data_stored":452797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":451879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":454147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":451469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888111,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":451207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1568768,"data_stored":1043029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:07:21.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T23:07:21.784 INFO:tasks.cephadm:Setting up client nodes... 2024-03-18T23:07:21.785 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T23:07:22.057 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:22.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:21 smithi132 ceph-mon[32020]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:22.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:21 smithi132 ceph-mon[32020]: pgmap v289: 97 pgs: 97 active+clean; 600 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-03-18T23:07:22.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:21 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/920376102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:22.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:21 smithi196 ceph-mon[67089]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:22.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:21 smithi196 ceph-mon[67089]: pgmap v289: 97 pgs: 97 active+clean; 600 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-03-18T23:07:22.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:21 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/920376102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:21 smithi144 ceph-mon[66781]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:07:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:21 smithi144 ceph-mon[66781]: pgmap v289: 97 pgs: 97 active+clean; 600 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-03-18T23:07:22.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:21 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/920376102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:23.156 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:22 smithi132 ceph-mon[32020]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.156 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:22 smithi132 ceph-mon[32020]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:22 smithi196 ceph-mon[67089]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:22 smithi196 ceph-mon[67089]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:22 smithi144 ceph-mon[66781]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.260 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:22 smithi144 ceph-mon[66781]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 -- 172.21.15.132:0/3758391619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050100cb0 msgr2=0x7fa0501010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 --2- 172.21.15.132:0/3758391619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050100cb0 0x7fa0501010d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa040009990 tx=0x7fa040009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 -- 172.21.15.132:0/3758391619 shutdown_connections 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 --2- 172.21.15.132:0/3758391619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050102590 0x7fa050108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 --2- 172.21.15.132:0/3758391619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050100cb0 0x7fa0501010d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 --2- 172.21.15.132:0/3758391619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050101bd0 0x7fa050102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 -- 172.21.15.132:0/3758391619 >> 172.21.15.132:0/3758391619 conn(0x7fa0500fbb30 msgr2=0x7fa0500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 -- 172.21.15.132:0/3758391619 shutdown_connections 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.733+0000 7fa057fcc700 1 -- 172.21.15.132:0/3758391619 wait complete. 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 Processor -- start 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 -- start start 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 0x7fa05019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010bd30 con 0x7fa050102590 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010bbb0 con 0x7fa050100cb0 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa057fcc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010beb0 con 0x7fa050101bd0 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa055d68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:23.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa055567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa056569700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 0x7fa05019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa055d68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37548/0 (socket says 172.21.15.132:37548) 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa055d68700 1 -- 172.21.15.132:0/4027665137 learned_addr learned my addr 172.21.15.132:0/4027665137 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.734+0000 7fa055567700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:58952/0 (socket says 172.21.15.132:58952) 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 -- 172.21.15.132:0/4027665137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 msgr2=0x7fa0501984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 -- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 msgr2=0x7fa05019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 0x7fa05019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 -- 172.21.15.132:0/4027665137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0501a0120 con 0x7fa050101bd0 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055d68700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:23.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa056569700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 0x7fa05019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa055567700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d750 tx=0x7fa04c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c014030 con 0x7fa050101bd0 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa04c00fb70 con 0x7fa050101bd0 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0501a0410 con 0x7fa050101bd0 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010d80 con 0x7fa050101bd0 2024-03-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.735+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501a0950 con 0x7fa050101bd0 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.736+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050050820 con 0x7fa050101bd0 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.737+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fa04c017030 con 0x7fa050101bd0 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.737+0000 7fa046ffd700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 0x7fa03c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.738+0000 7fa055d68700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 0x7fa03c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.738+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa04c0ce890 con 0x7fa050101bd0 2024-03-18T23:07:23.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.738+0000 7fa055d68700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 0x7fa03c0a4e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b2e0 tx=0x7fa040005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:23.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:23.741+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa04c09b880 con 0x7fa050101bd0 2024-03-18T23:07:24.019 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.017+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa050062f20 con 0x7fa050101bd0 2024-03-18T23:07:24.019 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:23 smithi132 ceph-mon[32020]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.2 KiB/s wr, 20 op/s 2024-03-18T23:07:24.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.021+0000 7fa046ffd700 1 -- 172.21.15.132:0/4027665137 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa04c09b180 con 0x7fa050101bd0 2024-03-18T23:07:24.022 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-03-18T23:07:24.022 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAsyfhlsp4fARAAd5lPQ+w6CcOiB6zYkRK7AA== 2024-03-18T23:07:24.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 msgr2=0x7fa03c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 0x7fa03c0a4e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b2e0 tx=0x7fa040005fd0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 msgr2=0x7fa050198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d750 tx=0x7fa04c00da60 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 shutdown_connections 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa03c0a29d0 0x7fa03c0a4e90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050100cb0 0x7fa0501984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa050102590 0x7fa05019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 --2- 172.21.15.132:0/4027665137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa050101bd0 0x7fa050198a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 >> 172.21.15.132:0/4027665137 conn(0x7fa0500fbb30 msgr2=0x7fa050104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 shutdown_connections 2024-03-18T23:07:24.025 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:24.024+0000 7fa057fcc700 1 -- 172.21.15.132:0/4027665137 wait complete. 2024-03-18T23:07:24.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:23 smithi196 ceph-mon[67089]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.2 KiB/s wr, 20 op/s 2024-03-18T23:07:24.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:23 smithi144 ceph-mon[66781]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.2 KiB/s wr, 20 op/s 2024-03-18T23:07:24.676 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:07:24.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-18T23:07:24.676 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T23:07:24.719 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T23:07:24.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-18T23:07:24.720 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-18T23:07:25.002 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:25.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:24 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4027665137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:24 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.116 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:24 smithi132 ceph-mon[32020]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:07:25.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:24 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4027665137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:24 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:24 smithi196 ceph-mon[67089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:07:25.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:24 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4027665137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:24 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T23:07:25.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:24 smithi144 ceph-mon[66781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T23:07:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:25 smithi132 ceph-mon[32020]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-18T23:07:26.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:25 smithi196 ceph-mon[67089]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-18T23:07:26.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:25 smithi144 ceph-mon[66781]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-03-18T23:07:26.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3172024834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0107250 msgr2=0x7f45e0107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3172024834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0107250 0x7f45e0107630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009990 tx=0x7f45d8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3172024834 shutdown_connections 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3172024834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101bb0 0x7f45e010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3172024834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0107250 0x7f45e0107630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3172024834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0101210 0x7f45e0101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3172024834 >> 172.21.15.132:0/3172024834 conn(0x7f45e0078140 msgr2=0x7f45e0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:26.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3172024834 shutdown_connections 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.648+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3172024834 wait complete. 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 Processor -- start 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 -- start start 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101210 0x7f45e019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0107250 0x7f45e0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:26.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111490 con 0x7f45e0101bb0 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111310 con 0x7f45e0107250 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e7fbb700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111610 con 0x7f45e0101210 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51448/0 (socket says 172.21.15.132:51448) 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 -- 172.21.15.132:0/3040792902 learned_addr learned my addr 172.21.15.132:0/3040792902 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5d57700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101210 0x7f45e019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:26.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e6558700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0107250 0x7f45e0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 -- 172.21.15.132:0/3040792902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101210 msgr2=0x7f45e019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101210 0x7f45e019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 -- 172.21.15.132:0/3040792902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0107250 msgr2=0x7f45e0198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0107250 0x7f45e0198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.649+0000 7f45e5556700 1 -- 172.21.15.132:0/3040792902 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e0199020 con 0x7f45e0101bb0 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45e5556700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45d400ef10 tx=0x7f45d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:26.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d400c9c0 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e01992b0 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d4019070 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4007e60 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.650+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e0102df0 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.651+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e0102900 con 0x7f45e0101bb0 2024-03-18T23:07:26.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.652+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f45d4012050 con 0x7f45e0101bb0 2024-03-18T23:07:26.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.653+0000 7f45deffd700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 0x7f45c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:26.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.653+0000 7f45e5d57700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 0x7f45c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:26.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.653+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f45d40ce9d0 con 0x7f45e0101bb0 2024-03-18T23:07:26.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.653+0000 7f45e5d57700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 0x7f45c80a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45d800b2e0 tx=0x7f45d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:26.659 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.657+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f45d4010040 con 0x7f45e0101bb0 2024-03-18T23:07:26.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.911+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f45e0062fc0 con 0x7f45e0101bb0 2024-03-18T23:07:26.916 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.916+0000 7f45deffd700 1 -- 172.21.15.132:0/3040792902 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262670 (secure 0 0 0) 0x7f45d409bac0 con 0x7f45e0101bb0 2024-03-18T23:07:26.918 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 msgr2=0x7f45c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 0x7f45c80a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45d800b2e0 tx=0x7f45d8005960 comp rx=0 tx=0).stop 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 msgr2=0x7f45e019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45d400ef10 tx=0x7f45d400c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 shutdown_connections 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f45c80a2920 0x7f45c80a4de0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45e0107250 0x7f45e0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45e0101210 0x7f45e019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 --2- 172.21.15.132:0/3040792902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0101bb0 0x7f45e019ec00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:26.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 >> 172.21.15.132:0/3040792902 conn(0x7f45e0078140 msgr2=0x7f45e0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:26.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 shutdown_connections 2024-03-18T23:07:26.923 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:26.919+0000 7f45e7fbb700 1 -- 172.21.15.132:0/3040792902 wait complete. 2024-03-18T23:07:27.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:26 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3040792902' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:07:27.237 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:26 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3040792902' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:07:27.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:26 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3040792902' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T23:07:27.616 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":3730787943},{"type":"v1","addr":"172.21.15.132:6801","nonce":3730787943}]},"active_addr":"172.21.15.132:6801/3730787943","active_change":"2024-03-18T23:00:18.961944+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.132:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":3187950145}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":3928210624}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":4032784803}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":2898017683}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":4139527659}]}]} 2024-03-18T23:07:27.621 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-18T23:07:27.621 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-18T23:07:27.622 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:07:27.896 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:28.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:27 smithi196 ceph-mon[67089]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-03-18T23:07:28.233 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:27 smithi132 ceph-mon[32020]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-03-18T23:07:28.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:27 smithi144 ceph-mon[66781]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.525+0000 7f13960fd700 1 -- 172.21.15.132:0/2522470370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1390100d10 msgr2=0x7f1390101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.525+0000 7f13960fd700 1 --2- 172.21.15.132:0/2522470370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1390100d10 0x7f1390101190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f13780099c0 tx=0x7f1378009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 -- 172.21.15.132:0/2522470370 shutdown_connections 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 --2- 172.21.15.132:0/2522470370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13901016d0 0x7f139010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 --2- 172.21.15.132:0/2522470370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 0x7f13901077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 --2- 172.21.15.132:0/2522470370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1390100d10 0x7f1390101190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 -- 172.21.15.132:0/2522470370 >> 172.21.15.132:0/2522470370 conn(0x7f1390076560 msgr2=0x7f1390076970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 -- 172.21.15.132:0/2522470370 shutdown_connections 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 -- 172.21.15.132:0/2522470370 wait complete. 2024-03-18T23:07:29.527 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 Processor -- start 2024-03-18T23:07:29.528 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.526+0000 7f13960fd700 1 -- start start 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 0x7f139019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 0x7f13901988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f13901126a0 con 0x7f13901016d0 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1390112520 con 0x7f1390100d10 2024-03-18T23:07:29.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f13960fd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1390112820 con 0x7f13901073e0 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 0x7f139019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51464/0 (socket says 172.21.15.132:51464) 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f1386ffd700 1 -- 172.21.15.132:0/1328245976 learned_addr learned my addr 172.21.15.132:0/1328245976 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138ffff700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 0x7f13901988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 -- 172.21.15.132:0/1328245976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 msgr2=0x7f13901988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 0x7f13901988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 -- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 msgr2=0x7f139019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 -- 172.21.15.132:0/1328245976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390199020 con 0x7f1390100d10 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f1386ffd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.527+0000 7f138f7fe700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 0x7f139019e700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9f0 tx=0x7f138000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:29.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.528+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380004220 con 0x7f1390100d10 2024-03-18T23:07:29.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.528+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1390199310 con 0x7f1390100d10 2024-03-18T23:07:29.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.528+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1380004d10 con 0x7f1390100d10 2024-03-18T23:07:29.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.528+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380005020 con 0x7f1390100d10 2024-03-18T23:07:29.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.528+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901a55d0 con 0x7f1390100d10 2024-03-18T23:07:29.531 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.529+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1390068fd0 con 0x7f1390100d10 2024-03-18T23:07:29.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.530+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f1380004750 con 0x7f1390100d10 2024-03-18T23:07:29.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.531+0000 7f138d7fa700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 0x7f137c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:29.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.531+0000 7f1386ffd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 0x7f137c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:29.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.531+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f13800ce9e0 con 0x7f1390100d10 2024-03-18T23:07:29.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.531+0000 7f1386ffd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 0x7f137c0a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13900769f0 tx=0x7f1378005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:29.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.539+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f138009bb60 con 0x7f1390100d10 2024-03-18T23:07:29.737 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.737+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13901a5860 con 0x7f1390100d10 2024-03-18T23:07:29.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.738+0000 7f138d7fa700 1 -- 172.21.15.132:0/1328245976 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22266 (secure 0 0 0) 0x7f138009b460 con 0x7f1390100d10 2024-03-18T23:07:29.739 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:29.739 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:07:15.045318+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 msgr2=0x7f137c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 0x7f137c0a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13900769f0 tx=0x7f1378005960 comp rx=0 tx=0).stop 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 msgr2=0x7f139019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 0x7f139019e700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9f0 tx=0x7f138000ed00 comp rx=0 tx=0).stop 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 shutdown_connections 2024-03-18T23:07:29.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f137c0a2920 0x7f137c0a4de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1390100d10 0x7f139019e700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13901073e0 0x7f13901988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 --2- 172.21.15.132:0/1328245976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901016d0 0x7f139019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 >> 172.21.15.132:0/1328245976 conn(0x7f1390076560 msgr2=0x7f13900fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 shutdown_connections 2024-03-18T23:07:29.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:29.741+0000 7f13960fd700 1 -- 172.21.15.132:0/1328245976 wait complete. 2024-03-18T23:07:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:30 smithi144 ceph-mon[66781]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-18T23:07:30.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:30 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1328245976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:30.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:30 smithi132 ceph-mon[32020]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-18T23:07:30.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:30 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1328245976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:30.422 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-18T23:07:30.423 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T23:07:30.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:30 smithi196 ceph-mon[67089]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-18T23:07:30.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:30 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1328245976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:30.664 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 -- 172.21.15.132:0/752554479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640101210 msgr2=0x7f9640101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 --2- 172.21.15.132:0/752554479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640101210 0x7f9640101670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9634009990 tx=0x7f9634009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 -- 172.21.15.132:0/752554479 shutdown_connections 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 --2- 172.21.15.132:0/752554479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101bb0 0x7f964010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 --2- 172.21.15.132:0/752554479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640107250 0x7f9640107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 --2- 172.21.15.132:0/752554479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640101210 0x7f9640101670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 -- 172.21.15.132:0/752554479 >> 172.21.15.132:0/752554479 conn(0x7f9640078140 msgr2=0x7f9640078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 -- 172.21.15.132:0/752554479 shutdown_connections 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.204+0000 7f9646c6b700 1 -- 172.21.15.132:0/752554479 wait complete. 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 Processor -- start 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 -- start start 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 0x7f9640197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 0x7f9640198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f96401114b0 con 0x7f9640101210 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9640111330 con 0x7f9640107250 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9646c6b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9640111630 con 0x7f9640101bb0 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9644a07700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f963ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 0x7f9640197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9644a07700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39234/0 (socket says 172.21.15.132:39234) 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9644a07700 1 -- 172.21.15.132:0/611760353 learned_addr learned my addr 172.21.15.132:0/611760353 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.205+0000 7f9645208700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 0x7f9640198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 -- 172.21.15.132:0/611760353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 msgr2=0x7f9640197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 0x7f9640197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 -- 172.21.15.132:0/611760353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 msgr2=0x7f9640198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 0x7f9640198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 -- 172.21.15.132:0/611760353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401a6f50 con 0x7f9640101210 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9645208700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 0x7f9640198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f963ffff700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 0x7f9640197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9644a07700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f963800ef30 tx=0x7f963800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380092e0 con 0x7f9640101210 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401a71e0 con 0x7f9640101210 2024-03-18T23:07:32.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638004bb0 con 0x7f9640101210 2024-03-18T23:07:32.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638004280 con 0x7f9640101210 2024-03-18T23:07:32.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.206+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a76f0 con 0x7f9640101210 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.207+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640068fd0 con 0x7f9640101210 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.208+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f963801b030 con 0x7f9640101210 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.209+0000 7f963dffb700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 0x7f96280a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.209+0000 7f963ffff700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 0x7f96280a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.209+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f96380ce510 con 0x7f9640101210 2024-03-18T23:07:32.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.209+0000 7f963ffff700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 0x7f96280a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f963400b2e0 tx=0x7f9634005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:32.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.213+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9638022020 con 0x7f9640101210 2024-03-18T23:07:32.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:32 smithi144 ceph-mon[66781]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-18T23:07:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:32 smithi132 ceph-mon[32020]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-18T23:07:32.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.409+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f964002dbb0 con 0x7f9640101210 2024-03-18T23:07:32.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.411+0000 7f963dffb700 1 -- 172.21.15.132:0/611760353 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22266 (secure 0 0 0) 0x7f9638010040 con 0x7f9640101210 2024-03-18T23:07:32.412 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:07:32.412 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":88,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:07:15.045318+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 msgr2=0x7f96280a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 0x7f96280a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f963400b2e0 tx=0x7f9634005960 comp rx=0 tx=0).stop 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 msgr2=0x7f96401976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f963800ef30 tx=0x7f963800c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 shutdown_connections 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2920 0x7f96280a4de0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9640107250 0x7f9640198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9640101210 0x7f96401976e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 --2- 172.21.15.132:0/611760353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9640101bb0 0x7f9640197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:32.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 >> 172.21.15.132:0/611760353 conn(0x7f9640078140 msgr2=0x7f96400ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:32.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 shutdown_connections 2024-03-18T23:07:32.415 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:32.413+0000 7f9646c6b700 1 -- 172.21.15.132:0/611760353 wait complete. 2024-03-18T23:07:32.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:32 smithi196 ceph-mon[67089]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-18T23:07:33.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:33 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/611760353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:33.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-18T23:07:33.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-18T23:07:33.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-18T23:07:33.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-18T23:07:33.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-18T23:07:33.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-18T23:07:33.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-18T23:07:33.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-18T23:07:33.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-18T23:07:33.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-18T23:07:33.169 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-18T23:07:33.169 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-18T23:07:33.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:33 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/611760353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:33.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:33 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/611760353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:07:33.844 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:33.853 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:33.863 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:33.959 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.305 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:34 smithi132 ceph-mon[32020]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-03-18T23:07:34.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:34 smithi144 ceph-mon[66781]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-03-18T23:07:34.391 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.392 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.392 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.393 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.407 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:34.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:34 smithi196 ceph-mon[67089]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-03-18T23:07:35.236 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:35.237 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:07:36.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:36 smithi132 ceph-mon[32020]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-18T23:07:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:36 smithi196 ceph-mon[67089]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-18T23:07:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:36 smithi144 ceph-mon[66781]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-18T23:07:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:38 smithi132 ceph-mon[32020]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-03-18T23:07:38.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:38 smithi196 ceph-mon[67089]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-03-18T23:07:38.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:38 smithi144 ceph-mon[66781]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-03-18T23:07:40.360 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:40 smithi132 ceph-mon[32020]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-18T23:07:40.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 -- 172.21.15.132:0/1276912507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 msgr2=0x7f46c8102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/1276912507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c8102930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f46b80099c0 tx=0x7f46b8009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 -- 172.21.15.132:0/1276912507 shutdown_connections 2024-03-18T23:07:40.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/1276912507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46c8107630 0x7f46c810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/1276912507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c8102930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/1276912507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8102f00 0x7f46c8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 -- 172.21.15.132:0/1276912507 >> 172.21.15.132:0/1276912507 conn(0x7f46c80fddc0 msgr2=0x7f46c81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 -- 172.21.15.132:0/1276912507 shutdown_connections 2024-03-18T23:07:40.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.462+0000 7f46cc9a2700 1 -- 172.21.15.132:0/1276912507 wait complete. 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 Processor -- start 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 -- start start 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46c8102f00 0x7f46c819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8107630 0x7f46c81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8112630 con 0x7f46c8102550 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81124b0 con 0x7f46c8102f00 2024-03-18T23:07:40.466 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46cc9a2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81127b0 con 0x7f46c8107630 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46c659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46c659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:39254/0 (socket says 172.21.15.132:39254) 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.463+0000 7f46c659c700 1 -- 172.21.15.132:0/3023422439 learned_addr learned my addr 172.21.15.132:0/3023422439 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c6d9d700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8107630 0x7f46c81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46bffff700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46c8102f00 0x7f46c819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 -- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8107630 msgr2=0x7f46c81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8107630 0x7f46c81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 -- 172.21.15.132:0/3023422439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46c8102f00 msgr2=0x7f46c819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46c8102f00 0x7f46c819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c8199030 con 0x7f46c8102550 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c6d9d700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c8107630 0x7f46c81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46c659c700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c819e710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006010 tx=0x7f46b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:40.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003680 con 0x7f46c8102550 2024-03-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c81992c0 con 0x7f46c8102550 2024-03-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b80050d0 con 0x7f46c8102550 2024-03-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8003db0 con 0x7f46c8102550 2024-03-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.464+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a55e0 con 0x7f46c8102550 2024-03-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.465+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46c8062aa0 con 0x7f46c8102550 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.466+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f46b8005280 con 0x7f46c8102550 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bf7fe700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b40a2ac0 0x7f46b40a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bffff700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b40a2ac0 0x7f46b40a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f46b80cfd20 con 0x7f46c8102550 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bf7fe700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] conn(0x7f46b40ac170 0x7f46b40ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46b8015ec0 con 0x7f46b40ac170 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f46b80d0200 con 0x7f46c8102550 2024-03-18T23:07:40.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.467+0000 7f46bffff700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b40a2ac0 0x7f46b40a4f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46c80ff1e0 tx=0x7f46b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:40.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.468+0000 7f46c6d9d700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] conn(0x7f46b40ac170 0x7f46b40ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:40.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.468+0000 7f46c6d9d700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] conn(0x7f46b40ac170 0x7f46b40ae590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:40.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.469+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== osd.2 v2:172.21.15.196:6800/1879525213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46b8015ec0 con 0x7f46b40ac170 2024-03-18T23:07:40.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:40 smithi196 ceph-mon[67089]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-18T23:07:40.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.485+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 --> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46c8062ea0 con 0x7f46b40ac170 2024-03-18T23:07:40.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.485+0000 7f46bf7fe700 1 -- 172.21.15.132:0/3023422439 <== osd.2 v2:172.21.15.196:6800/1879525213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46c8062ea0 con 0x7f46b40ac170 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] conn(0x7f46b40ac170 msgr2=0x7f46b40ae590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] conn(0x7f46b40ac170 0x7f46b40ae590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b40a2ac0 msgr2=0x7f46b40a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f46b40a2ac0 0x7f46b40a4f80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46c80ff1e0 tx=0x7f46b0009260 comp rx=0 tx=0).stop 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 msgr2=0x7f46c819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 --2- 172.21.15.132:0/3023422439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c8102550 0x7f46c819e710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006010 tx=0x7f46b800f690 comp rx=0 tx=0).stop 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46c6d9d700 1 -- 172.21.15.132:0/3023422439 reap_dead start 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 shutdown_connections 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 >> 172.21.15.132:0/3023422439 conn(0x7f46c80fddc0 msgr2=0x7f46c80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:40.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 shutdown_connections 2024-03-18T23:07:40.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:40.486+0000 7f46cc9a2700 1 -- 172.21.15.132:0/3023422439 wait complete. 2024-03-18T23:07:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:40 smithi144 ceph-mon[66781]: pgmap v298: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-18T23:07:42.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:42 smithi132 ceph-mon[32020]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-18T23:07:42.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:42 smithi196 ceph-mon[67089]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-18T23:07:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:42 smithi144 ceph-mon[66781]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- 172.21.15.132:0/2719060850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb060108390 msgr2=0x7fb060108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 --2- 172.21.15.132:0/2719060850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb060108390 0x7fb060108770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb0500099c0 tx=0x7fb050009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- 172.21.15.132:0/2719060850 shutdown_connections 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 --2- 172.21.15.132:0/2719060850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006b3b0 0x7fb06010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 --2- 172.21.15.132:0/2719060850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb060108390 0x7fb060108770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 --2- 172.21.15.132:0/2719060850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb06006aa10 0x7fb06006ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.818 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- 172.21.15.132:0/2719060850 >> 172.21.15.132:0/2719060850 conn(0x7fb060077230 msgr2=0x7fb060077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- 172.21.15.132:0/2719060850 shutdown_connections 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- 172.21.15.132:0/2719060850 wait complete. 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 Processor -- start 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.785+0000 7fb066806700 1 -- start start 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 0x7fb0601a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 0x7fb0601a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060108390 0x7fb06019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:43.819 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601125d0 con 0x7fb06006b3b0 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb060112450 con 0x7fb06006aa10 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb066806700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb060112750 con 0x7fb060108390 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 0x7fb0601a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 0x7fb0601a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 0x7fb0601a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53548/0 (socket says 172.21.15.132:53548) 2024-03-18T23:07:43.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 0x7fb0601a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38822/0 (socket says 172.21.15.132:38822) 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05f7fe700 1 -- 172.21.15.132:0/1261235059 learned_addr learned my addr 172.21.15.132:0/1261235059 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb064da3700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060108390 0x7fb06019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 -- 172.21.15.132:0/1261235059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060108390 msgr2=0x7fb06019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb060108390 0x7fb06019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 -- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 msgr2=0x7fb0601a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 0x7fb0601a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019b220 con 0x7fb06006aa10 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05f7fe700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb06006b3b0 0x7fb0601a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.786+0000 7fb05ffff700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 0x7fb0601a0900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b580 tx=0x7fb050003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:43.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050003680 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06019b4b0 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0500055a0 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a77d0 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000fc90 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.787+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb06010df80 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fb05000f460 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb0480a2ac0 0x7fb0480a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05f7fe700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb0480a2ac0 0x7fb0480a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb0500cff30 con 0x7fb06006aa10 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] conn(0x7fb0480ac2a0 0x7fb0480ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:43.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb050003680 con 0x7fb0480ac2a0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.789+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb0500d02e0 con 0x7fb06006aa10 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.790+0000 7fb05f7fe700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb0480a2ac0 0x7fb0480a4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb060103340 tx=0x7fb054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.790+0000 7fb064da3700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] conn(0x7fb0480ac2a0 0x7fb0480ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.790+0000 7fb064da3700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] conn(0x7fb0480ac2a0 0x7fb0480ae6c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.791+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== osd.4 v2:172.21.15.144:6808/962231389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb050003680 con 0x7fb0480ac2a0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.807+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 --> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb060103a40 con 0x7fb0480ac2a0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.807+0000 7fb05d7fa700 1 -- 172.21.15.132:0/1261235059 <== osd.4 v2:172.21.15.144:6808/962231389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb060103a40 con 0x7fb0480ac2a0 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] conn(0x7fb0480ac2a0 msgr2=0x7fb0480ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] conn(0x7fb0480ac2a0 0x7fb0480ae6c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb0480a2ac0 msgr2=0x7fb0480a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb0480a2ac0 0x7fb0480a4f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb060103340 tx=0x7fb054009260 comp rx=0 tx=0).stop 2024-03-18T23:07:43.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 msgr2=0x7fb0601a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 --2- 172.21.15.132:0/1261235059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06006aa10 0x7fb0601a0900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b580 tx=0x7fb050003a90 comp rx=0 tx=0).stop 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb064da3700 1 -- 172.21.15.132:0/1261235059 reap_dead start 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 shutdown_connections 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 >> 172.21.15.132:0/1261235059 conn(0x7fb060077230 msgr2=0x7fb06010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 shutdown_connections 2024-03-18T23:07:43.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:43.808+0000 7fb066806700 1 -- 172.21.15.132:0/1261235059 wait complete. 2024-03-18T23:07:44.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:44 smithi132 ceph-mon[32020]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:44.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:44 smithi196 ceph-mon[67089]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:44 smithi144 ceph-mon[66781]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 -- 172.21.15.132:0/3974655493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6050101e50 msgr2=0x7f6050105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 --2- 172.21.15.132:0/3974655493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6050101e50 0x7f6050105e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60400099c0 tx=0x7f6040009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 -- 172.21.15.132:0/3974655493 shutdown_connections 2024-03-18T23:07:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 --2- 172.21.15.132:0/3974655493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60501064f0 0x7f605010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 --2- 172.21.15.132:0/3974655493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60501014a0 0x7f6050101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 --2- 172.21.15.132:0/3974655493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6050101e50 0x7f6050105e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 -- 172.21.15.132:0/3974655493 >> 172.21.15.132:0/3974655493 conn(0x7f6050077260 msgr2=0x7f6050077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 -- 172.21.15.132:0/3974655493 shutdown_connections 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.147+0000 7f6058305700 1 -- 172.21.15.132:0/3974655493 wait complete. 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 Processor -- start 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 -- start start 2024-03-18T23:07:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 0x7f605019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050101e50 0x7f605019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60501064f0 0x7f60501977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f60501114d0 con 0x7f60501014a0 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6050111350 con 0x7f60501064f0 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f6058305700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6050111650 con 0x7f6050101e50 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.148+0000 7f60560a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 0x7f605019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 0x7f605019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53558/0 (socket says 172.21.15.132:53558) 2024-03-18T23:07:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 -- 172.21.15.132:0/2438913646 learned_addr learned my addr 172.21.15.132:0/2438913646 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:46.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60558a0700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050101e50 0x7f605019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60568a2700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60501064f0 0x7f60501977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 -- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050101e50 msgr2=0x7f605019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6050101e50 0x7f605019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 -- 172.21.15.132:0/2438913646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60501064f0 msgr2=0x7f60501977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60501064f0 0x7f60501977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6050197f90 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60560a1700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 0x7f605019d630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f604c00ead0 tx=0x7f604c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c00b9c0 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6050198280 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f604c004d10 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c005700 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.149+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605006a850 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.150+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f605006ad80 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.151+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f604c004750 con 0x7f60501014a0 2024-03-18T23:07:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60477fe700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f603c0a2920 0x7f603c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60558a0700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f603c0a2920 0x7f603c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f604c0d08c0 con 0x7f60501014a0 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60477fe700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] conn(0x7f603c0ac100 0x7f603c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f604c011e50 con 0x7f603c0ac100 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f604c0d0da0 con 0x7f60501014a0 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60558a0700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f603c0a2920 0x7f603c0a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f6040005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.152+0000 7f60568a2700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] conn(0x7f603c0ac100 0x7f603c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.153+0000 7f60568a2700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] conn(0x7f603c0ac100 0x7f603c0ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.153+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== osd.11 v2:172.21.15.196:6824/946882152 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f604c011e50 con 0x7f603c0ac100 2024-03-18T23:07:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.169+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 --> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f605006b200 con 0x7f603c0ac100 2024-03-18T23:07:46.171 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f60477fe700 1 -- 172.21.15.132:0/2438913646 <== osd.11 v2:172.21.15.196:6824/946882152 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f605006b200 con 0x7f603c0ac100 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] conn(0x7f603c0ac100 msgr2=0x7f603c0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] conn(0x7f603c0ac100 0x7f603c0ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f603c0a2920 msgr2=0x7f603c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f603c0a2920 0x7f603c0a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f6040005fb0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 msgr2=0x7f605019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 --2- 172.21.15.132:0/2438913646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60501014a0 0x7f605019d630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f604c00ead0 tx=0x7f604c00ede0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f60568a2700 1 -- 172.21.15.132:0/2438913646 reap_dead start 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 shutdown_connections 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.170+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 >> 172.21.15.132:0/2438913646 conn(0x7f6050077260 msgr2=0x7f60500ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.171+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 shutdown_connections 2024-03-18T23:07:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.171+0000 7f6058305700 1 -- 172.21.15.132:0/2438913646 wait complete. 2024-03-18T23:07:46.240 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:46 smithi132 ceph-mon[32020]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 -- 172.21.15.132:0/943552247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102360 msgr2=0x7fc58c1027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 --2- 172.21.15.132:0/943552247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102360 0x7fc58c1027e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5800099c0 tx=0x7fc580009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 -- 172.21.15.132:0/943552247 shutdown_connections 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 --2- 172.21.15.132:0/943552247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 0x7fc58c108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 --2- 172.21.15.132:0/943552247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102d20 0x7fc58c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 --2- 172.21.15.132:0/943552247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102360 0x7fc58c1027e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 -- 172.21.15.132:0/943552247 >> 172.21.15.132:0/943552247 conn(0x7fc58c079440 msgr2=0x7fc58c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:46.383 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 -- 172.21.15.132:0/943552247 shutdown_connections 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.340+0000 7fc591204700 1 -- 172.21.15.132:0/943552247 wait complete. 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.341+0000 7fc591204700 1 Processor -- start 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.341+0000 7fc591204700 1 -- start start 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 0x7fc58c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 0x7fc58c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 0x7fc58c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c1125c0 con 0x7fc58c102d20 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c112440 con 0x7fc58c108340 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c112740 con 0x7fc58c102360 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 0x7fc58c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 0x7fc58c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 0x7fc58c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.384 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 0x7fc58c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53578/0 (socket says 172.21.15.132:53578) 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58ad9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 0x7fc58c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:44564/0 (socket says 172.21.15.132:44564) 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58a59c700 1 -- 172.21.15.132:0/195575485 learned_addr learned my addr 172.21.15.132:0/195575485 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 -- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 msgr2=0x7fc58c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 0x7fc58c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 -- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 msgr2=0x7fc58c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 0x7fc58c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c199040 con 0x7fc58c108340 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58a59c700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c102d20 0x7fc58c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58ad9d700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc58c102360 0x7fc58c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc58b59e700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 0x7fc58c1988e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc58400efc0 tx=0x7fc58400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5840092e0 con 0x7fc58c108340 2024-03-18T23:07:46.385 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc584004bb0 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c1992d0 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584004280 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.342+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c1a55a0 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.343+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc58c062aa0 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.344+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fc58401b030 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc573fff700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc5740a2920 0x7fc5740a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc58ad9d700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc5740a2920 0x7fc5740a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc5840ce780 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc573fff700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] conn(0x7fc5740ac100 0x7fc5740ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5740aebd0 con 0x7fc5740ac100 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc5840ceb30 con 0x7fc58c108340 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc58ad9d700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc5740a2920 0x7fc5740a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc58c100b20 tx=0x7fc57c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc58a59c700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] conn(0x7fc5740ac100 0x7fc5740ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.345+0000 7fc58a59c700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] conn(0x7fc5740ac100 0x7fc5740ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.346+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== osd.5 v2:172.21.15.196:6808/1616570087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5740aebd0 con 0x7fc5740ac100 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.362+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 --> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc58c062f20 con 0x7fc5740ac100 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.362+0000 7fc573fff700 1 -- 172.21.15.132:0/195575485 <== osd.5 v2:172.21.15.196:6808/1616570087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc58c062f20 con 0x7fc5740ac100 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.362+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] conn(0x7fc5740ac100 msgr2=0x7fc5740ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.362+0000 7fc591204700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] conn(0x7fc5740ac100 0x7fc5740ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc5740a2920 msgr2=0x7fc5740a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fc5740a2920 0x7fc5740a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc58c100b20 tx=0x7fc57c009260 comp rx=0 tx=0).stop 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 msgr2=0x7fc58c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 --2- 172.21.15.132:0/195575485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108340 0x7fc58c1988e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc58400efc0 tx=0x7fc58400c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc58b59e700 1 -- 172.21.15.132:0/195575485 reap_dead start 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 shutdown_connections 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 >> 172.21.15.132:0/195575485 conn(0x7fc58c079440 msgr2=0x7fc58c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 shutdown_connections 2024-03-18T23:07:46.388 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:46.363+0000 7fc591204700 1 -- 172.21.15.132:0/195575485 wait complete. 2024-03-18T23:07:46.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:46 smithi196 ceph-mon[67089]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-18T23:07:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:46 smithi144 ceph-mon[66781]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-18T23:07:48.302 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:48 smithi132 ceph-mon[32020]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-18T23:07:48.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:48 smithi196 ceph-mon[67089]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- 172.21.15.132:0/1674722096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81006b5c0 msgr2=0x7fe810106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 --2- 172.21.15.132:0/1674722096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81006b5c0 0x7fe810106f50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80099c0 tx=0x7fe7f8009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- 172.21.15.132:0/1674722096 shutdown_connections 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 --2- 172.21.15.132:0/1674722096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 0x7fe81006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 --2- 172.21.15.132:0/1674722096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe810107680 0x7fe81010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 --2- 172.21.15.132:0/1674722096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81006b5c0 0x7fe810106f50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- 172.21.15.132:0/1674722096 >> 172.21.15.132:0/1674722096 conn(0x7fe810078180 msgr2=0x7fe810078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- 172.21.15.132:0/1674722096 shutdown_connections 2024-03-18T23:07:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- 172.21.15.132:0/1674722096 wait complete. 2024-03-18T23:07:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 Processor -- start 2024-03-18T23:07:48.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.606+0000 7fe815950700 1 -- start start 2024-03-18T23:07:48.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:48 smithi144 ceph-mon[66781]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-18T23:07:48.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 0x7fe81019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:48.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe81006b5c0 0x7fe81019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:48.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe810107680 0x7fe8101988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:48.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe8101125f0 con 0x7fe810107680 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe810112470 con 0x7fe81006ac10 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe815950700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe810112770 con 0x7fe81006b5c0 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe810107680 0x7fe8101988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 0x7fe81019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe810107680 0x7fe8101988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53592/0 (socket says 172.21.15.132:53592) 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80f7fe700 1 -- 172.21.15.132:0/862406752 learned_addr learned my addr 172.21.15.132:0/862406752 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe8067fc700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe81006b5c0 0x7fe81019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 -- 172.21.15.132:0/862406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe81006b5c0 msgr2=0x7fe81019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe81006b5c0 0x7fe81019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 -- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe810107680 msgr2=0x7fe8101988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe810107680 0x7fe8101988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe810199050 con 0x7fe81006ac10 2024-03-18T23:07:48.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.607+0000 7fe80effd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 0x7fe81019e730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe80000e9f0 tx=0x7fe80000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.608+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80000c9c0 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.608+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80000cb60 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.608+0000 7fe815950700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe810199340 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.608+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800018680 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.608+0000 7fe815950700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8101a5600 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.609+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe810062aa0 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.610+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fe800010430 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe81494e700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7fc0a2920 0x7fe7fc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe8067fc700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7fc0a2920 0x7fe7fc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe8000cf080 con 0x7fe81006ac10 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe81494e700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] conn(0x7fe7fc0ac100 0x7fe7fc0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe80001bec0 con 0x7fe7fc0ac100 2024-03-18T23:07:48.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe8000cf4e0 con 0x7fe81006ac10 2024-03-18T23:07:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe8067fc700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7fc0a2920 0x7fe7fc0a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009990 tx=0x7fe7f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.611+0000 7fe80f7fe700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] conn(0x7fe7fc0ac100 0x7fe7fc0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.612+0000 7fe80f7fe700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] conn(0x7fe7fc0ac100 0x7fe7fc0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:48.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.612+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== osd.6 v2:172.21.15.132:6818/428682768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe80001bec0 con 0x7fe7fc0ac100 2024-03-18T23:07:48.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.628+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 --> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe810062ea0 con 0x7fe7fc0ac100 2024-03-18T23:07:48.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.628+0000 7fe81494e700 1 -- 172.21.15.132:0/862406752 <== osd.6 v2:172.21.15.132:6818/428682768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe810062ea0 con 0x7fe7fc0ac100 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] conn(0x7fe7fc0ac100 msgr2=0x7fe7fc0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] conn(0x7fe7fc0ac100 0x7fe7fc0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7fc0a2920 msgr2=0x7fe7fc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7fc0a2920 0x7fe7fc0a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009990 tx=0x7fe7f8005fb0 comp rx=0 tx=0).stop 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 msgr2=0x7fe81019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 --2- 172.21.15.132:0/862406752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81006ac10 0x7fe81019e730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe80000e9f0 tx=0x7fe80000ed00 comp rx=0 tx=0).stop 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe80f7fe700 1 -- 172.21.15.132:0/862406752 reap_dead start 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 shutdown_connections 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 >> 172.21.15.132:0/862406752 conn(0x7fe810078180 msgr2=0x7fe8100ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 shutdown_connections 2024-03-18T23:07:48.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:48.629+0000 7fe806ffd700 1 -- 172.21.15.132:0/862406752 wait complete. 2024-03-18T23:07:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:50 smithi132 ceph-mon[32020]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:50.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:50 smithi196 ceph-mon[67089]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:50.571 INFO:teuthology.orchestra.run.smithi132.stdout:171798691889 2024-03-18T23:07:50.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-18T23:07:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:50 smithi144 ceph-mon[66781]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:50.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 -- 172.21.15.132:0/4040329280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406b460 msgr2=0x7f8214106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 --2- 172.21.15.132:0/4040329280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406b460 0x7f8214106f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f8204009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 -- 172.21.15.132:0/4040329280 shutdown_connections 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 --2- 172.21.15.132:0/4040329280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406aab0 0x7f821406ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 --2- 172.21.15.132:0/4040329280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8214107620 0x7f821410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.761+0000 7f821b444700 1 --2- 172.21.15.132:0/4040329280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406b460 0x7f8214106f40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.762+0000 7f821b444700 1 -- 172.21.15.132:0/4040329280 >> 172.21.15.132:0/4040329280 conn(0x7f8214077260 msgr2=0x7f8214077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.762+0000 7f821b444700 1 -- 172.21.15.132:0/4040329280 shutdown_connections 2024-03-18T23:07:50.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.762+0000 7f821b444700 1 -- 172.21.15.132:0/4040329280 wait complete. 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.762+0000 7f821b444700 1 Processor -- start 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 -- start start 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 0x7f821419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406b460 0x7f821419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8214107620 0x7f82141988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f82141125c0 con 0x7f821406aab0 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8214112440 con 0x7f821406b460 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f821b444700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8214112740 con 0x7f8214107620 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82189df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406b460 0x7f821419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 0x7f821419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82189df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406b460 0x7f821419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38908/0 (socket says 172.21.15.132:38908) 2024-03-18T23:07:50.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82189df700 1 -- 172.21.15.132:0/2885324768 learned_addr learned my addr 172.21.15.132:0/2885324768 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 0x7f821419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53602/0 (socket says 172.21.15.132:53602) 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 -- 172.21.15.132:0/2885324768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8214107620 msgr2=0x7f82141988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8214107620 0x7f82141988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 -- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406b460 msgr2=0x7f821419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821406b460 0x7f821419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.763+0000 7f82191e0700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8214199020 con 0x7f821406aab0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f82191e0700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 0x7f821419e6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000cbd0 con 0x7f821406aab0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821000cd70 con 0x7f821406aab0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210018840 con 0x7f821406aab0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8214199310 con 0x7f821406aab0 2024-03-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.764+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141a5580 con 0x7f821406aab0 2024-03-18T23:07:50.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.765+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8214062a60 con 0x7f821406aab0 2024-03-18T23:07:50.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f82100189e0 con 0x7f821406aab0 2024-03-18T23:07:50.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f82000a2920 0x7f82000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f82100cf420 con 0x7f821406aab0 2024-03-18T23:07:50.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f82189df700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f82000a2920 0x7f82000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] conn(0x7f82000ac100 0x7f82000ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82000aebd0 con 0x7f82000ac100 2024-03-18T23:07:50.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.767+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f82100cf900 con 0x7f821406aab0 2024-03-18T23:07:50.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.768+0000 7f82199e1700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] conn(0x7f82000ac100 0x7f82000ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.768+0000 7f82189df700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f82000a2920 0x7f82000a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8214108a80 tx=0x7f820401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.768+0000 7f82199e1700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] conn(0x7f82000ac100 0x7f82000ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.769+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== osd.1 v2:172.21.15.144:6800/1173848453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82000aebd0 con 0x7f82000ac100 2024-03-18T23:07:50.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.786+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 --> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8214062ee0 con 0x7f82000ac100 2024-03-18T23:07:50.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.786+0000 7f820a7fc700 1 -- 172.21.15.132:0/2885324768 <== osd.1 v2:172.21.15.144:6800/1173848453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8214062ee0 con 0x7f82000ac100 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] conn(0x7f82000ac100 msgr2=0x7f82000ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] conn(0x7f82000ac100 0x7f82000ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f82000a2920 msgr2=0x7f82000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f82000a2920 0x7f82000a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8214108a80 tx=0x7f820401a040 comp rx=0 tx=0).stop 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 msgr2=0x7f821419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 --2- 172.21.15.132:0/2885324768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f821406aab0 0x7f821419e6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000edb0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f82199e1700 1 -- 172.21.15.132:0/2885324768 reap_dead start 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 shutdown_connections 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 >> 172.21.15.132:0/2885324768 conn(0x7f8214077260 msgr2=0x7f82140ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 shutdown_connections 2024-03-18T23:07:50.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.787+0000 7f821b444700 1 -- 172.21.15.132:0/2885324768 wait complete. 2024-03-18T23:07:50.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 -- 172.21.15.132:0/1084888726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 msgr2=0x7f2928107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 --2- 172.21.15.132:0/1084888726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928107680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2918009990 tx=0x7f2918009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 -- 172.21.15.132:0/1084888726 shutdown_connections 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 --2- 172.21.15.132:0/1084888726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2928101bb0 0x7f292810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 --2- 172.21.15.132:0/1084888726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928107680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 --2- 172.21.15.132:0/1084888726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29281011f0 0x7f2928101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 -- 172.21.15.132:0/1084888726 >> 172.21.15.132:0/1084888726 conn(0x7f2928078220 msgr2=0x7f2928078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 -- 172.21.15.132:0/1084888726 shutdown_connections 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.984+0000 7f292d57c700 1 -- 172.21.15.132:0/1084888726 wait complete. 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 Processor -- start 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 -- start start 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 0x7f2928102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 0x7f29281033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.993 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29281114e0 con 0x7f29281072a0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2928111360 con 0x7f2928101bb0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f292d57c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2928111660 con 0x7f29281011f0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f29277fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f2926ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 0x7f2928102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f29267fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 0x7f29281033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f2926ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 0x7f2928102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:44612/0 (socket says 172.21.15.132:44612) 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f29277fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53620/0 (socket says 172.21.15.132:53620) 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f29267fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 0x7f29281033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38936/0 (socket says 172.21.15.132:38936) 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.985+0000 7f2926ffd700 1 -- 172.21.15.132:0/93051707 learned_addr learned my addr 172.21.15.132:0/93051707 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 -- 172.21.15.132:0/93051707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 msgr2=0x7f29281033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 0x7f29281033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 -- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 msgr2=0x7f2928103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29281a6f80 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f29267fc700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2928101bb0 0x7f29281033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f29277fe700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29281072a0 0x7f2928103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f2926ffd700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 0x7f2928102ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2918000c00 tx=0x7f291800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918027070 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281a7210 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2918003d70 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800f460 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.986+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281a7740 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.987+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2928106300 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.988+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f291802a070 con 0x7f29281011f0 2024-03-18T23:07:50.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.988+0000 7f290ffff700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f29100a2920 0x7f29100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f29267fc700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f29100a2920 0x7f29100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f29180cff20 con 0x7f29281011f0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f290ffff700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] conn(0x7f29100ac100 0x7f29100ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29100aebd0 con 0x7f29100ac100 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f29180d02d0 con 0x7f29281011f0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f29277fe700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] conn(0x7f29100ac100 0x7f29100ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f29267fc700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f29100a2920 0x7f29100a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2928078820 tx=0x7f291c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.989+0000 7f29277fe700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] conn(0x7f29100ac100 0x7f29100ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:50.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:50.990+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== osd.3 v2:172.21.15.132:6810/3584795430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29100aebd0 con 0x7f29100ac100 2024-03-18T23:07:51.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.007+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 --> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29281028e0 con 0x7f29100ac100 2024-03-18T23:07:51.008 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f290ffff700 1 -- 172.21.15.132:0/93051707 <== osd.3 v2:172.21.15.132:6810/3584795430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f29281028e0 con 0x7f29100ac100 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] conn(0x7f29100ac100 msgr2=0x7f29100ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] conn(0x7f29100ac100 0x7f29100ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f29100a2920 msgr2=0x7f29100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f29100a2920 0x7f29100a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2928078820 tx=0x7f291c009260 comp rx=0 tx=0).stop 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 msgr2=0x7f2928102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:51.009 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 --2- 172.21.15.132:0/93051707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29281011f0 0x7f2928102ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2918000c00 tx=0x7f291800bab0 comp rx=0 tx=0).stop 2024-03-18T23:07:51.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f29277fe700 1 -- 172.21.15.132:0/93051707 reap_dead start 2024-03-18T23:07:51.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 shutdown_connections 2024-03-18T23:07:51.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 >> 172.21.15.132:0/93051707 conn(0x7f2928078220 msgr2=0x7f292806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:51.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 shutdown_connections 2024-03-18T23:07:51.010 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:51.008+0000 7f292d57c700 1 -- 172.21.15.132:0/93051707 wait complete. 2024-03-18T23:07:51.431 INFO:teuthology.orchestra.run.smithi132.stdout:146028888117 2024-03-18T23:07:51.432 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-18T23:07:51.655 INFO:teuthology.orchestra.run.smithi132.stdout:356482285588 2024-03-18T23:07:51.656 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-18T23:07:52.142 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 -- 172.21.15.132:0/2206924973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1064f0 msgr2=0x7fbc2c10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 --2- 172.21.15.132:0/2206924973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1064f0 0x7fbc2c10e0c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbc240099c0 tx=0x7fbc24009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 -- 172.21.15.132:0/2206924973 shutdown_connections 2024-03-18T23:07:52.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 --2- 172.21.15.132:0/2206924973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1064f0 0x7fbc2c10e0c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 --2- 172.21.15.132:0/2206924973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 --2- 172.21.15.132:0/2206924973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 0x7fbc2c105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 -- 172.21.15.132:0/2206924973 >> 172.21.15.132:0/2206924973 conn(0x7fbc2c077260 msgr2=0x7fbc2c077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:52.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.142+0000 7fbc347e5700 1 -- 172.21.15.132:0/2206924973 shutdown_connections 2024-03-18T23:07:52.144 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.143+0000 7fbc347e5700 1 -- 172.21.15.132:0/2206924973 wait complete. 2024-03-18T23:07:52.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 Processor -- start 2024-03-18T23:07:52.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 -- start start 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 0x7fbc2c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1064f0 0x7fbc2c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111500 con 0x7fbc2c1014a0 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111380 con 0x7fbc2c101e50 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc347e5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111680 con 0x7fbc2c1064f0 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c19d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43556/0 (socket says 172.21.15.132:43556) 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 -- 172.21.15.132:0/1563103500 learned_addr learned my addr 172.21.15.132:0/1563103500 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc31d80700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 0x7fbc2c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:52.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32d82700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1064f0 0x7fbc2c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 -- 172.21.15.132:0/1563103500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1064f0 msgr2=0x7fbc2c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c1064f0 0x7fbc2c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 -- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 msgr2=0x7fbc2c19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 0x7fbc2c19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c197f80 con 0x7fbc2c1014a0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc31d80700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c101e50 0x7fbc2c19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.144+0000 7fbc32581700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c19d620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800ef30 tx=0x7fbc2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.145+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280092e0 con 0x7fbc2c1014a0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.145+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc28004bb0 con 0x7fbc2c1014a0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.145+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c198210 con 0x7fbc2c1014a0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.145+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28004280 con 0x7fbc2c1014a0 2024-03-18T23:07:52.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.145+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a4480 con 0x7fbc2c1014a0 2024-03-18T23:07:52.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.146+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc2c062aa0 con 0x7fbc2c1014a0 2024-03-18T23:07:52.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.147+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fbc2801b030 con 0x7fbc2c1014a0 2024-03-18T23:07:52.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.147+0000 7fbc237fe700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbc180a2920 0x7fbc180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:52.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fbc280cef20 con 0x7fbc2c1014a0 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc237fe700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] conn(0x7fbc180ac100 0x7fbc180ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc31d80700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbc180a2920 0x7fbc180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc2801fe50 con 0x7fbc180ac100 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbc280cf400 con 0x7fbc2c1014a0 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.148+0000 7fbc32d82700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] conn(0x7fbc180ac100 0x7fbc180ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:52.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.149+0000 7fbc31d80700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbc180a2920 0x7fbc180a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c1078c0 tx=0x7fbc1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:52.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.149+0000 7fbc32d82700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] conn(0x7fbc180ac100 0x7fbc180ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:52.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.149+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== osd.7 v2:172.21.15.144:6816/3972442848 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc2801fe50 con 0x7fbc180ac100 2024-03-18T23:07:52.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.165+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 --> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc2c062f20 con 0x7fbc180ac100 2024-03-18T23:07:52.166 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc237fe700 1 -- 172.21.15.132:0/1563103500 <== osd.7 v2:172.21.15.144:6816/3972442848 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbc2c062f20 con 0x7fbc180ac100 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] conn(0x7fbc180ac100 msgr2=0x7fbc180ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] conn(0x7fbc180ac100 0x7fbc180ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbc180a2920 msgr2=0x7fbc180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbc180a2920 0x7fbc180a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c1078c0 tx=0x7fbc1c009260 comp rx=0 tx=0).stop 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 msgr2=0x7fbc2c19d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 --2- 172.21.15.132:0/1563103500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c1014a0 0x7fbc2c19d620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800ef30 tx=0x7fbc2800c3a0 comp rx=0 tx=0).stop 2024-03-18T23:07:52.167 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc32d82700 1 -- 172.21.15.132:0/1563103500 reap_dead start 2024-03-18T23:07:52.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 shutdown_connections 2024-03-18T23:07:52.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 >> 172.21.15.132:0/1563103500 conn(0x7fbc2c077260 msgr2=0x7fbc2c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:52.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 shutdown_connections 2024-03-18T23:07:52.168 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:52.166+0000 7fbc347e5700 1 -- 172.21.15.132:0/1563103500 wait complete. 2024-03-18T23:07:52.177 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:52 smithi132 ceph-mon[32020]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:52.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:52 smithi196 ceph-mon[67089]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:52 smithi144 ceph-mon[66781]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:07:54.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:54 smithi132 ceph-mon[32020]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:07:54.393 INFO:teuthology.orchestra.run.smithi132.stdout:111669149756 2024-03-18T23:07:54.394 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-18T23:07:54.410 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:54 smithi144 ceph-mon[66781]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:07:54.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:54 smithi196 ceph-mon[67089]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:07:54.839 INFO:teuthology.orchestra.run.smithi132.stdout:231928234025 2024-03-18T23:07:54.839 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-18T23:07:55.241 INFO:teuthology.orchestra.run.smithi132.stdout:77309411391 2024-03-18T23:07:55.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 -- 172.21.15.132:0/3976717340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514108410 msgr2=0x7f85141087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 --2- 172.21.15.132:0/3976717340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514108410 0x7f85141087f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8510009990 tx=0x7f8510009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 -- 172.21.15.132:0/3976717340 shutdown_connections 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 --2- 172.21.15.132:0/3976717340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514102d20 0x7f851410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 --2- 172.21.15.132:0/3976717340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514108410 0x7f85141087f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 --2- 172.21.15.132:0/3976717340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8514102360 0x7f85141027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 -- 172.21.15.132:0/3976717340 >> 172.21.15.132:0/3976717340 conn(0x7f8514079440 msgr2=0x7f8514079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.538+0000 7f851c113700 1 -- 172.21.15.132:0/3976717340 shutdown_connections 2024-03-18T23:07:55.540 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 -- 172.21.15.132:0/3976717340 wait complete. 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 Processor -- start 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 -- start start 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8514102360 0x7f8514074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 0x7f8514074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514108410 0x7f8514075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8514112650 con 0x7f8514102d20 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85141124d0 con 0x7f8514108410 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851c113700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85141127d0 con 0x7f8514102360 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f85196ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 0x7f8514074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f85196ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 0x7f8514074b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43564/0 (socket says 172.21.15.132:43564) 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f85196ae700 1 -- 172.21.15.132:0/3160020924 learned_addr learned my addr 172.21.15.132:0/3160020924 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f8519eaf700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8514102360 0x7f8514074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.539+0000 7f851a6b0700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514108410 0x7f8514075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 -- 172.21.15.132:0/3160020924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8514102360 msgr2=0x7f8514074610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8514102360 0x7f8514074610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 -- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514108410 msgr2=0x7f8514075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514108410 0x7f8514075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85141a7010 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f851a6b0700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8514108410 0x7f8514075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f85196ae700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 0x7f8514074b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f850400e9c0 tx=0x7f850400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850400cbd0 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141a7300 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850400ee60 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504005630 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.540+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85141a77f0 con 0x7f8514102d20 2024-03-18T23:07:55.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.541+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85140629e0 con 0x7f8514102d20 2024-03-18T23:07:55.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.543+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8504004750 con 0x7f8514102d20 2024-03-18T23:07:55.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f850affd700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f85000a2920 0x7f85000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f8519eaf700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f85000a2920 0x7f85000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f85040cf100 con 0x7f8514102d20 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f850affd700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] conn(0x7f85000ac100 0x7f85000ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8504017e50 con 0x7f85000ac100 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f85040cf5e0 con 0x7f8514102d20 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f851a6b0700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] conn(0x7f85000ac100 0x7f85000ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f8519eaf700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f85000a2920 0x7f85000a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f851000b2e0 tx=0x7f8510005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.544+0000 7f851a6b0700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] conn(0x7f85000ac100 0x7f85000ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.545+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== osd.10 v2:172.21.15.144:6824/2873915844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8504017e50 con 0x7f85000ac100 2024-03-18T23:07:55.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 --> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8514062e60 con 0x7f85000ac100 2024-03-18T23:07:55.562 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f850affd700 1 -- 172.21.15.132:0/3160020924 <== osd.10 v2:172.21.15.144:6824/2873915844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8514062e60 con 0x7f85000ac100 2024-03-18T23:07:55.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] conn(0x7f85000ac100 msgr2=0x7f85000ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.563 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] conn(0x7f85000ac100 0x7f85000ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f85000a2920 msgr2=0x7f85000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f85000a2920 0x7f85000a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f851000b2e0 tx=0x7f8510005960 comp rx=0 tx=0).stop 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 msgr2=0x7f8514074b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.562+0000 7f851c113700 1 --2- 172.21.15.132:0/3160020924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514102d20 0x7f8514074b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f850400e9c0 tx=0x7f850400ecd0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.563+0000 7f851a6b0700 1 -- 172.21.15.132:0/3160020924 reap_dead start 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.563+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 shutdown_connections 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.563+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 >> 172.21.15.132:0/3160020924 conn(0x7f8514079440 msgr2=0x7f8514100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.563+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 shutdown_connections 2024-03-18T23:07:55.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.563+0000 7f851c113700 1 -- 172.21.15.132:0/3160020924 wait complete. 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 -- 172.21.15.132:0/1859717012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c108470 msgr2=0x7f793c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 --2- 172.21.15.132:0/1859717012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c108470 0x7f793c108850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79300098f0 tx=0x7f7930009c00 comp rx=0 tx=0).stop 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 -- 172.21.15.132:0/1859717012 shutdown_connections 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 --2- 172.21.15.132:0/1859717012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c108470 0x7f793c108850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 --2- 172.21.15.132:0/1859717012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.754 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 --2- 172.21.15.132:0/1859717012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f793c102390 0x7f793c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 -- 172.21.15.132:0/1859717012 >> 172.21.15.132:0/1859717012 conn(0x7f793c0793d0 msgr2=0x7f793c0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:55.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 -- 172.21.15.132:0/1859717012 shutdown_connections 2024-03-18T23:07:55.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.753+0000 7f7940c34700 1 -- 172.21.15.132:0/1859717012 wait complete. 2024-03-18T23:07:55.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 Processor -- start 2024-03-18T23:07:55.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 -- start start 2024-03-18T23:07:55.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c102390 0x7f793c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f793c108470 0x7f793c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f793c112620 con 0x7f793c102d50 2024-03-18T23:07:55.757 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f793c1124a0 con 0x7f793c102390 2024-03-18T23:07:55.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.754+0000 7f7940c34700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f793c1127a0 con 0x7f793c108470 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7939d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7939d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43582/0 (socket says 172.21.15.132:43582) 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c102390 0x7f793c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7939d9b700 1 -- 172.21.15.132:0/1620506678 learned_addr learned my addr 172.21.15.132:0/1620506678 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793ad9d700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f793c108470 0x7f793c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 -- 172.21.15.132:0/1620506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f793c108470 msgr2=0x7f793c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f793c108470 0x7f793c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 -- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 msgr2=0x7f793c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1990a0 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7939d9b700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c102d50 0x7f793c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f793a59c700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c102390 0x7f793c19e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f793000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930005040 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79300056c0 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c199330 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930003a00 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.755+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c1a5650 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.756+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f793c062aa0 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.757+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f793000b560 con 0x7f793c102390 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f792b7fe700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f79240a29a0 0x7f79240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f7939d9b700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f79240a29a0 0x7f79240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f79300cfdf0 con 0x7f793c102390 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f792b7fe700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] conn(0x7f79240ac180 0x7f79240ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f793000b0b0 con 0x7f79240ac180 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f79300d01a0 con 0x7f793c102390 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f793ad9d700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] conn(0x7f79240ac180 0x7f79240ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f7939d9b700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f79240a29a0 0x7f79240a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f793c0799d0 tx=0x7f792c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.758+0000 7f793ad9d700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] conn(0x7f79240ac180 0x7f79240ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:55.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.759+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== osd.0 v2:172.21.15.132:6802/3523914432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f793000b0b0 con 0x7f79240ac180 2024-03-18T23:07:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.775+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 --> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f793c062ea0 con 0x7f79240ac180 2024-03-18T23:07:55.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.775+0000 7f792b7fe700 1 -- 172.21.15.132:0/1620506678 <== osd.0 v2:172.21.15.132:6802/3523914432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f793c062ea0 con 0x7f79240ac180 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] conn(0x7f79240ac180 msgr2=0x7f79240ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] conn(0x7f79240ac180 0x7f79240ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f79240a29a0 msgr2=0x7f79240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f79240a29a0 0x7f79240a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f793c0799d0 tx=0x7f792c009260 comp rx=0 tx=0).stop 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c102390 msgr2=0x7f793c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 --2- 172.21.15.132:0/1620506678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c102390 0x7f793c19e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f793000fa00 comp rx=0 tx=0).stop 2024-03-18T23:07:55.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f793ad9d700 1 -- 172.21.15.132:0/1620506678 reap_dead start 2024-03-18T23:07:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 shutdown_connections 2024-03-18T23:07:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 >> 172.21.15.132:0/1620506678 conn(0x7f793c0793d0 msgr2=0x7f793c101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 shutdown_connections 2024-03-18T23:07:55.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:55.776+0000 7f7940c34700 1 -- 172.21.15.132:0/1620506678 wait complete. 2024-03-18T23:07:56.398 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:56 smithi132 ceph-mon[32020]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 -- 172.21.15.132:0/3633314394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102f20 msgr2=0x7f2be8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 --2- 172.21.15.132:0/3633314394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102f20 0x7f2be8106f70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80099c0 tx=0x7f2bd8009cd0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 -- 172.21.15.132:0/3633314394 shutdown_connections 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 --2- 172.21.15.132:0/3633314394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be8107650 0x7f2be810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 --2- 172.21.15.132:0/3633314394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102570 0x7f2be8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 --2- 172.21.15.132:0/3633314394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102f20 0x7f2be8106f70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 -- 172.21.15.132:0/3633314394 >> 172.21.15.132:0/3633314394 conn(0x7f2be80fdde0 msgr2=0x7f2be8100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 -- 172.21.15.132:0/3633314394 shutdown_connections 2024-03-18T23:07:56.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.417+0000 7f2befc45700 1 -- 172.21.15.132:0/3633314394 wait complete. 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 Processor -- start 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 -- start start 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102570 0x7f2be819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 0x7f2be819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be8107650 0x7f2be8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8112610 con 0x7f2be8102570 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8112490 con 0x7f2be8107650 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.418+0000 7f2befc45700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8112790 con 0x7f2be8102f20 2024-03-18T23:07:56.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 0x7f2be819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed9e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102570 0x7f2be819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 0x7f2be819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:34612/0 (socket says 172.21.15.132:34612) 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed9e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102570 0x7f2be819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43592/0 (socket says 172.21.15.132:43592) 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 -- 172.21.15.132:0/4129580830 learned_addr learned my addr 172.21.15.132:0/4129580830 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bee1e2700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be8107650 0x7f2be8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 -- 172.21.15.132:0/4129580830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be8107650 msgr2=0x7f2be8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be8107650 0x7f2be8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 -- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102570 msgr2=0x7f2be819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2be8102570 0x7f2be819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be8199090 con 0x7f2be8102f20 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bed1e0700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 0x7f2be819ec60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800fa00 tx=0x7f2bd800fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8027070 con 0x7f2be8102f20 2024-03-18T23:07:56.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be8199320 con 0x7f2be8102f20 2024-03-18T23:07:56.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8003680 con 0x7f2be8102f20 2024-03-18T23:07:56.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80173f0 con 0x7f2be8102f20 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.419+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a55f0 con 0x7f2be8102f20 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.420+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2be8062aa0 con 0x7f2be8102f20 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.421+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f2bd802a070 con 0x7f2be8102f20 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bdeffd700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2bd40a2920 0x7f2bd40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bed9e1700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2bd40a2920 0x7f2bd40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2bd80d0470 con 0x7f2be8102f20 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bdeffd700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] conn(0x7f2bd40ac100 0x7f2bd40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:56.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8009030 con 0x7f2bd40ac100 2024-03-18T23:07:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2bd80d0820 con 0x7f2be8102f20 2024-03-18T23:07:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.422+0000 7f2bee1e2700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] conn(0x7f2bd40ac100 0x7f2bd40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.423+0000 7f2bed9e1700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2bd40a2920 0x7f2bd40a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2be80fe1e0 tx=0x7f2be4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.423+0000 7f2bee1e2700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] conn(0x7f2bd40ac100 0x7f2bd40ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:56.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.424+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== osd.9 v2:172.21.15.132:6826/118980095 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8009030 con 0x7f2bd40ac100 2024-03-18T23:07:56.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.445+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 --> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2be8062ea0 con 0x7f2bd40ac100 2024-03-18T23:07:56.446 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.445+0000 7f2bdeffd700 1 -- 172.21.15.132:0/4129580830 <== osd.9 v2:172.21.15.132:6826/118980095 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2be8062ea0 con 0x7f2bd40ac100 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] conn(0x7f2bd40ac100 msgr2=0x7f2bd40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] conn(0x7f2bd40ac100 0x7f2bd40ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2bd40a2920 msgr2=0x7f2bd40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2bd40a2920 0x7f2bd40a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2be80fe1e0 tx=0x7f2be4009260 comp rx=0 tx=0).stop 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 msgr2=0x7f2be819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 --2- 172.21.15.132:0/4129580830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2be8102f20 0x7f2be819ec60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800fa00 tx=0x7f2bd800fa30 comp rx=0 tx=0).stop 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2bee1e2700 1 -- 172.21.15.132:0/4129580830 reap_dead start 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 shutdown_connections 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 >> 172.21.15.132:0/4129580830 conn(0x7f2be80fdde0 msgr2=0x7f2be8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 shutdown_connections 2024-03-18T23:07:56.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:56.446+0000 7f2befc45700 1 -- 172.21.15.132:0/4129580830 wait complete. 2024-03-18T23:07:56.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:56 smithi196 ceph-mon[67089]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:07:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:56 smithi144 ceph-mon[66781]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:07:56.944 INFO:teuthology.orchestra.run.smithi132.stdout:326417514523 2024-03-18T23:07:56.944 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-18T23:07:57.785 INFO:teuthology.orchestra.run.smithi132.stdout:206158430252 2024-03-18T23:07:57.786 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-18T23:07:57.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2622554059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 msgr2=0x7f3eb810b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2622554059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb810b090 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009990 tx=0x7f3ea8009ca0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.956 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2622554059 shutdown_connections 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2622554059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb8114260 0x7f3eb8116650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2622554059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb810b090 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2622554059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb810b5d0 0x7f3eb8113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2622554059 >> 172.21.15.132:0/2622554059 conn(0x7f3eb806d450 msgr2=0x7f3eb806d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2622554059 shutdown_connections 2024-03-18T23:07:57.957 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.955+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2622554059 wait complete. 2024-03-18T23:07:57.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.956+0000 7f3ebc83e700 1 Processor -- start 2024-03-18T23:07:57.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.956+0000 7f3ebc83e700 1 -- start start 2024-03-18T23:07:57.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.956+0000 7f3ebc83e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb81a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.956+0000 7f3ebc83e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb810b5d0 0x7f3eb81a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ebc83e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb8114260 0x7f3eb81a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ebc83e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8119d40 con 0x7f3eb810acb0 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ebc83e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8119bc0 con 0x7f3eb8114260 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ebc83e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8119ec0 con 0x7f3eb810b5d0 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb81a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb81a34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43612/0 (socket says 172.21.15.132:43612) 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 -- 172.21.15.132:0/2461258221 learned_addr learned my addr 172.21.15.132:0/2461258221 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb5d9b700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb810b5d0 0x7f3eb81a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb6d9d700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb8114260 0x7f3eb81a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 -- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb810b5d0 msgr2=0x7f3eb81a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb810b5d0 0x7f3eb81a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 -- 172.21.15.132:0/2461258221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb8114260 msgr2=0x7f3eb81a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb8114260 0x7f3eb81a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb81a8490 con 0x7f3eb810acb0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb5d9b700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3eb810b5d0 0x7f3eb81a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3eb659c700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb81a34d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8000c00 tx=0x7f3ea800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8027070 con 0x7f3eb810acb0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb81a8720 con 0x7f3eb810acb0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.957+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8003d70 con 0x7f3eb810acb0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.958+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800f460 con 0x7f3eb810acb0 2024-03-18T23:07:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.958+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81b0000 con 0x7f3eb810acb0 2024-03-18T23:07:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.958+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3eb8062aa0 con 0x7f3eb810acb0 2024-03-18T23:07:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.960+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3ea802a070 con 0x7f3eb810acb0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.960+0000 7f3ea77fe700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ea00a2920 0x7f3ea00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.960+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3ea80d00d0 con 0x7f3eb810acb0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.960+0000 7f3eb5d9b700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ea00a2920 0x7f3ea00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3ea77fe700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] conn(0x7f3ea00ac100 0x7f3ea00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ea00aebd0 con 0x7f3ea00ac100 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80d05b0 con 0x7f3eb810acb0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3eb6d9d700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] conn(0x7f3ea00ac100 0x7f3ea00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3eb5d9b700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ea00a2920 0x7f3ea00a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009680 tx=0x7f3eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.961+0000 7f3eb6d9d700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] conn(0x7f3ea00ac100 0x7f3ea00ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:07:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.962+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== osd.8 v2:172.21.15.196:6816/3559751523 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ea00aebd0 con 0x7f3ea00ac100 2024-03-18T23:07:57.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.978+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 --> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3eb8062ea0 con 0x7f3ea00ac100 2024-03-18T23:07:57.979 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ea77fe700 1 -- 172.21.15.132:0/2461258221 <== osd.8 v2:172.21.15.196:6816/3559751523 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3eb8062ea0 con 0x7f3ea00ac100 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] conn(0x7f3ea00ac100 msgr2=0x7f3ea00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] conn(0x7f3ea00ac100 0x7f3ea00ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ea00a2920 msgr2=0x7f3ea00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3ea00a2920 0x7f3ea00a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009680 tx=0x7f3eac009260 comp rx=0 tx=0).stop 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 msgr2=0x7f3eb81a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 --2- 172.21.15.132:0/2461258221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3eb810acb0 0x7f3eb81a34d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8000c00 tx=0x7f3ea800bab0 comp rx=0 tx=0).stop 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3eb6d9d700 1 -- 172.21.15.132:0/2461258221 reap_dead start 2024-03-18T23:07:57.980 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 shutdown_connections 2024-03-18T23:07:57.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 >> 172.21.15.132:0/2461258221 conn(0x7f3eb806d450 msgr2=0x7f3eb8109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:07:57.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 shutdown_connections 2024-03-18T23:07:57.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:07:57.979+0000 7f3ebc83e700 1 -- 172.21.15.132:0/2461258221 wait complete. 2024-03-18T23:07:58.438 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:07:58 smithi132 ceph-mon[32020]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:07:58.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:07:58 smithi196 ceph-mon[67089]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:07:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:07:58 smithi144 ceph-mon[66781]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:07:58.725 INFO:teuthology.orchestra.run.smithi132.stdout:55834574918 2024-03-18T23:07:58.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-18T23:07:59.303 INFO:teuthology.orchestra.run.smithi132.stdout:34359738441 2024-03-18T23:07:59.304 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-18T23:07:59.748 INFO:teuthology.orchestra.run.smithi132.stdout:300647710753 2024-03-18T23:07:59.749 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-18T23:08:00.299 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:00 smithi132 ceph-mon[32020]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:00.328 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:00.328 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:00.347 INFO:teuthology.orchestra.run.smithi132.stdout:266287972388 2024-03-18T23:08:00.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-18T23:08:00.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:00 smithi196 ceph-mon[67089]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:00 smithi144 ceph-mon[66781]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:02.302 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:02 smithi132 ceph-mon[32020]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-18T23:08:02.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:02 smithi196 ceph-mon[67089]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-18T23:08:02.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- 172.21.15.132:0/1213010288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 msgr2=0x7f728010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1213010288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728010f160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f727800b0d0 tx=0x7f727800b3e0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- 172.21.15.132:0/1213010288 shutdown_connections 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1213010288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728010f160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1213010288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f7280108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1213010288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 0x7f72801027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- 172.21.15.132:0/1213010288 >> 172.21.15.132:0/1213010288 conn(0x7f7280079440 msgr2=0x7f7280079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- 172.21.15.132:0/1213010288 shutdown_connections 2024-03-18T23:08:02.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- 172.21.15.132:0/1213010288 wait complete. 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 Processor -- start 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.532+0000 7f7286c5e700 1 -- start start 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 0x7f728019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f72801987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f72801125c0 con 0x7f7280102d20 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7280112440 con 0x7f7280102360 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7280112740 con 0x7f7280108340 2024-03-18T23:08:02.535 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33428/0 (socket says 172.21.15.132:33428) 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 -- 172.21.15.132:0/1577379842 learned_addr learned my addr 172.21.15.132:0/1577379842 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f72849fa700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 0x7f728019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f72851fb700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f72801987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 -- 172.21.15.132:0/1577379842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 msgr2=0x7f72801987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f72801987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 -- 172.21.15.132:0/1577379842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 msgr2=0x7f728019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 0x7f728019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 -- 172.21.15.132:0/1577379842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7280198f80 con 0x7f7280102d20 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f72851fb700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f72801987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727ffff700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f726c00eb30 tx=0x7f726c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:02.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c00cc90 con 0x7f7280102d20 2024-03-18T23:08:02.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7280199270 con 0x7f7280102d20 2024-03-18T23:08:02.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f726c004500 con 0x7f7280102d20 2024-03-18T23:08:02.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.533+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c010430 con 0x7f7280102d20 2024-03-18T23:08:02.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.534+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801a5480 con 0x7f7280102d20 2024-03-18T23:08:02.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.534+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72801039f0 con 0x7f7280102d20 2024-03-18T23:08:02.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.535+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f726c003f70 con 0x7f7280102d20 2024-03-18T23:08:02.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.536+0000 7f727dffb700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 0x7f72700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:02.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.536+0000 7f72849fa700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 0x7f72700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:02.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.536+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f726c0ceda0 con 0x7f7280102d20 2024-03-18T23:08:02.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.536+0000 7f72849fa700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 0x7f72700a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7274009680 tx=0x7f7274009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:02.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.541+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f726c09be90 con 0x7f7280102d20 2024-03-18T23:08:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:02 smithi144 ceph-mon[66781]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-18T23:08:02.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.741+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7280199cd0 con 0x7f7280102d20 2024-03-18T23:08:02.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.741+0000 7f727dffb700 1 -- 172.21.15.132:0/1577379842 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f726c09b790 con 0x7f7280102d20 2024-03-18T23:08:02.742 INFO:teuthology.orchestra.run.smithi132.stdout:171798691892 2024-03-18T23:08:02.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 msgr2=0x7f72700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:02.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 0x7f72700a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7274009680 tx=0x7f7274009260 comp rx=0 tx=0).stop 2024-03-18T23:08:02.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 msgr2=0x7f728019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f726c00eb30 tx=0x7f726c00ee40 comp rx=0 tx=0).stop 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 shutdown_connections 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f72700a2920 0x7f72700a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7280102360 0x7f728019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7280108340 0x7f72801987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 --2- 172.21.15.132:0/1577379842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7280102d20 0x7f728019eb60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 >> 172.21.15.132:0/1577379842 conn(0x7f7280079440 msgr2=0x7f7280100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 shutdown_connections 2024-03-18T23:08:02.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:02.744+0000 7f7286c5e700 1 -- 172.21.15.132:0/1577379842 wait complete. 2024-03-18T23:08:03.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:03 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1577379842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:08:03.419 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.419 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.420 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.422 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.422 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.423 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.424 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:03.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:03 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1577379842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:08:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:03 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1577379842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T23:08:03.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691889 got 171798691892 for osd.5 2024-03-18T23:08:03.842 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:04.053 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 -- 172.21.15.132:0/1185007572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8107250 msgr2=0x7fefa8107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 --2- 172.21.15.132:0/1185007572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8107250 0x7fefa8107630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009f90 tx=0x7fefa000b330 comp rx=0 tx=0).stop 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 -- 172.21.15.132:0/1185007572 shutdown_connections 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 --2- 172.21.15.132:0/1185007572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8107250 0x7fefa8107630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 --2- 172.21.15.132:0/1185007572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 --2- 172.21.15.132:0/1185007572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8101210 0x7fefa8101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 -- 172.21.15.132:0/1185007572 >> 172.21.15.132:0/1185007572 conn(0x7fefa8078140 msgr2=0x7fefa8078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 -- 172.21.15.132:0/1185007572 shutdown_connections 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.286+0000 7fefaf165700 1 -- 172.21.15.132:0/1185007572 wait complete. 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 Processor -- start 2024-03-18T23:08:04.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 -- start start 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 0x7fefa819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8107250 0x7fefa81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8111490 con 0x7fefa8101bb0 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8111310 con 0x7fefa8101210 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefaf165700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8111610 con 0x7fefa8107250 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefa7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefa7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33452/0 (socket says 172.21.15.132:33452) 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefa7fff700 1 -- 172.21.15.132:0/4102840967 learned_addr learned my addr 172.21.15.132:0/4102840967 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefacf01700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 0x7fefa819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.287+0000 7fefad702700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8107250 0x7fefa81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 -- 172.21.15.132:0/4102840967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8107250 msgr2=0x7fefa81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8107250 0x7fefa81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 -- 172.21.15.132:0/4102840967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 msgr2=0x7fefa819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 0x7fefa819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 -- 172.21.15.132:0/4102840967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa8199000 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefacf01700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 0x7fefa819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa7fff700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e9a0 tx=0x7fef9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00ca10 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81992f0 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c00cbb0 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c018680 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.288+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a55b0 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.289+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8068fd0 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.290+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fef9c010430 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.291+0000 7fefa5ffb700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 0x7fef900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.291+0000 7fefacf01700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 0x7fef900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.291+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fef9c0cec30 con 0x7fefa8101bb0 2024-03-18T23:08:04.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.291+0000 7fefacf01700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 0x7fef900a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009790 tx=0x7fefa0009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:04.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.295+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef9c09bd20 con 0x7fefa8101bb0 2024-03-18T23:08:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:04 smithi196 ceph-mon[67089]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-18T23:08:04.476 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:04 smithi132 ceph-mon[32020]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-18T23:08:04.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.513+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fefa81a57e0 con 0x7fefa8101bb0 2024-03-18T23:08:04.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.513+0000 7fefa5ffb700 1 -- 172.21.15.132:0/4102840967 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fef9c09b620 con 0x7fefa8101bb0 2024-03-18T23:08:04.515 INFO:teuthology.orchestra.run.smithi132.stdout:146028888121 2024-03-18T23:08:04.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 msgr2=0x7fef900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 0x7fef900a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009790 tx=0x7fefa0009720 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 msgr2=0x7fefa819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e9a0 tx=0x7fef9c00ecb0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 shutdown_connections 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fef900a2920 0x7fef900a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefa8101210 0x7fefa819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefa8107250 0x7fefa81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 --2- 172.21.15.132:0/4102840967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa8101bb0 0x7fefa819ec20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 >> 172.21.15.132:0/4102840967 conn(0x7fefa8078140 msgr2=0x7fefa8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 shutdown_connections 2024-03-18T23:08:04.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:04.516+0000 7fefaf165700 1 -- 172.21.15.132:0/4102840967 wait complete. 2024-03-18T23:08:04.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:04 smithi144 ceph-mon[66781]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-03-18T23:08:05.283 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:05 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4102840967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:08:05.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:05 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4102840967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:08:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:05 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4102840967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T23:08:06.310 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:06 smithi132 ceph-mon[32020]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:06 smithi196 ceph-mon[67089]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:06.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:06 smithi144 ceph-mon[66781]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:08.239 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:08.326 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:08 smithi132 ceph-mon[32020]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:08.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:08 smithi196 ceph-mon[67089]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:08 smithi144 ceph-mon[66781]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:08.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888117 got 146028888121 for osd.4 2024-03-18T23:08:08.649 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:10.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:10 smithi132 ceph-mon[32020]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:10 smithi196 ceph-mon[67089]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:10 smithi144 ceph-mon[66781]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:12.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:12 smithi196 ceph-mon[67089]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:12.562 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:12 smithi132 ceph-mon[32020]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:12.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:12 smithi144 ceph-mon[66781]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:13.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 -- 172.21.15.132:0/1874952196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 msgr2=0x7f992c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:13.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 --2- 172.21.15.132:0/1874952196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c102810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9918009990 tx=0x7f9918009ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 -- 172.21.15.132:0/1874952196 shutdown_connections 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 --2- 172.21.15.132:0/1874952196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 --2- 172.21.15.132:0/1874952196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 0x7f992c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 --2- 172.21.15.132:0/1874952196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c102810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.220+0000 7f9933085700 1 -- 172.21.15.132:0/1874952196 >> 172.21.15.132:0/1874952196 conn(0x7f992c0793d0 msgr2=0x7f992c0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.221+0000 7f9933085700 1 -- 172.21.15.132:0/1874952196 shutdown_connections 2024-03-18T23:08:13.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.221+0000 7f9933085700 1 -- 172.21.15.132:0/1874952196 wait complete. 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.221+0000 7f9933085700 1 Processor -- start 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.221+0000 7f9933085700 1 -- start start 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 0x7f992c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1125d0 con 0x7f992c102390 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f992c112450 con 0x7f992c102d50 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f992c112750 con 0x7f992c108470 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9930e21700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9930e21700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40812/0 (socket says 172.21.15.132:40812) 2024-03-18T23:08:13.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9931622700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 0x7f992c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9930e21700 1 -- 172.21.15.132:0/310307792 learned_addr learned my addr 172.21.15.132:0/310307792 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 -- 172.21.15.132:0/310307792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 msgr2=0x7f992c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 0x7f992c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 -- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 msgr2=0x7f992c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 -- 172.21.15.132:0/310307792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c19b410 con 0x7f992c102d50 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9930e21700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f992bfff700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c1a0fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99180059d0 tx=0x7f991800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918027070 con 0x7f992c102d50 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c19b6a0 con 0x7f992c102d50 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.222+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9918003d70 con 0x7f992c102d50 2024-03-18T23:08:13.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.223+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918022e30 con 0x7f992c102d50 2024-03-18T23:08:13.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.223+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1a7810 con 0x7f992c102d50 2024-03-18T23:08:13.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.224+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c103a50 con 0x7f992c102d50 2024-03-18T23:08:13.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.225+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f991802a070 con 0x7f992c102d50 2024-03-18T23:08:13.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.225+0000 7f9929ffb700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 0x7f991c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:13.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.225+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f99180cf8a0 con 0x7f992c102d50 2024-03-18T23:08:13.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.225+0000 7f9930e21700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 0x7f991c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:13.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.226+0000 7f9930e21700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 0x7f991c0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9920011550 tx=0x7f9920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:13.230 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.230+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9918013070 con 0x7f992c102d50 2024-03-18T23:08:13.440 INFO:teuthology.orchestra.run.smithi132.stdout:231928234029 2024-03-18T23:08:13.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.439+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f992c068fd0 con 0x7f992c102d50 2024-03-18T23:08:13.441 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.440+0000 7f9929ffb700 1 -- 172.21.15.132:0/310307792 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f991809c920 con 0x7f992c102d50 2024-03-18T23:08:13.442 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 msgr2=0x7f991c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 0x7f991c0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9920011550 tx=0x7f9920009260 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 msgr2=0x7f992c1a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c1a0fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99180059d0 tx=0x7f991800bab0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 shutdown_connections 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f991c0a2920 0x7f991c0a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f992c108470 0x7f992c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c102390 0x7f992c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 --2- 172.21.15.132:0/310307792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f992c102d50 0x7f992c1a0fa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 >> 172.21.15.132:0/310307792 conn(0x7f992c0793d0 msgr2=0x7f992c10d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 shutdown_connections 2024-03-18T23:08:13.443 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.442+0000 7f9933085700 1 -- 172.21.15.132:0/310307792 wait complete. 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 -- 172.21.15.132:0/430052204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0108340 msgr2=0x7f1af0108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 --2- 172.21.15.132:0/430052204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0108340 0x7f1af0108720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00099c0 tx=0x7f1ae0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 -- 172.21.15.132:0/430052204 shutdown_connections 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 --2- 172.21.15.132:0/430052204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102d20 0x7f1af010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 --2- 172.21.15.132:0/430052204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0108340 0x7f1af0108720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 --2- 172.21.15.132:0/430052204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0102360 0x7f1af01027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 -- 172.21.15.132:0/430052204 >> 172.21.15.132:0/430052204 conn(0x7f1af0079440 msgr2=0x7f1af0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 -- 172.21.15.132:0/430052204 shutdown_connections 2024-03-18T23:08:14.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.300+0000 7f1af597e700 1 -- 172.21.15.132:0/430052204 wait complete. 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 Processor -- start 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 -- start start 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 0x7f1af019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0108340 0x7f1af0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01125f0 con 0x7f1af0102d20 2024-03-18T23:08:14.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112470 con 0x7f1af0102360 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1af597e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112770 con 0x7f1af0108340 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1aee7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1aee7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40856/0 (socket says 172.21.15.132:40856) 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1aeeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 0x7f1af019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1aee7fc700 1 -- 172.21.15.132:0/780945869 learned_addr learned my addr 172.21.15.132:0/780945869 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.301+0000 7f1aef7fe700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0108340 0x7f1af0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 -- 172.21.15.132:0/780945869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0108340 msgr2=0x7f1af0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0108340 0x7f1af0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 -- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 msgr2=0x7f1af019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:14.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 -- 172.21.15.132:0/780945869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af0198ff0 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aee7fc700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1aeeffd700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 0x7f1af019e690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098e0 tx=0x7f1ae000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0027070 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0199280 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae0003680 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00173f0 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.302+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a54f0 con 0x7f1af0102360 2024-03-18T23:08:14.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.303+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af01039f0 con 0x7f1af0102360 2024-03-18T23:08:14.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.304+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f1ae002a070 con 0x7f1af0102360 2024-03-18T23:08:14.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.305+0000 7f1af497c700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 0x7f1ad80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:14.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.305+0000 7f1aee7fc700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 0x7f1ad80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:14.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.305+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1ae00cf760 con 0x7f1af0102360 2024-03-18T23:08:14.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.305+0000 7f1aee7fc700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 0x7f1ad80a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1af00799d0 tx=0x7f1ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:14.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.308+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ae0013070 con 0x7f1af0102360 2024-03-18T23:08:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:14 smithi144 ceph-mon[66781]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:14 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:14.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:14 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/310307792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:08:14.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:14 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3232981619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:08:14.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:14 smithi196 ceph-mon[67089]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:14 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:14 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/310307792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:08:14.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:14 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3232981619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:08:14.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.508+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1af0199d40 con 0x7f1af0102360 2024-03-18T23:08:14.509 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:14 smithi132 ceph-mon[32020]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:14.509 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:14 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:08:14.509 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:14 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/310307792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T23:08:14.509 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:14 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3232981619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T23:08:14.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.508+0000 7f1af497c700 1 -- 172.21.15.132:0/780945869 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ae009c8e0 con 0x7f1af0102360 2024-03-18T23:08:14.510 INFO:teuthology.orchestra.run.smithi132.stdout:77309411397 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 msgr2=0x7f1ad80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 0x7f1ad80a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1af00799d0 tx=0x7f1ae4009260 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 msgr2=0x7f1af019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 0x7f1af019e690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098e0 tx=0x7f1ae000fa00 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 shutdown_connections 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ad80a2920 0x7f1ad80a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af0102360 0x7f1af019e690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1af0108340 0x7f1af0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 --2- 172.21.15.132:0/780945869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af0102d20 0x7f1af019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:14.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 >> 172.21.15.132:0/780945869 conn(0x7f1af0079440 msgr2=0x7f1af0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:14.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 shutdown_connections 2024-03-18T23:08:14.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:14.512+0000 7f1af597e700 1 -- 172.21.15.132:0/780945869 wait complete. 2024-03-18T23:08:15.259 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:15 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/780945869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:08:15.259 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:15 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.259 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:15 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.260 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:15 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.260 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:15 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.285 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234025 got 231928234029 for osd.7 2024-03-18T23:08:15.285 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:15 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/780945869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:08:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:15 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:15 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:15 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:15 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1565039321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 msgr2=0x7f7e2410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1565039321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2410f220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b0a0 tx=0x7f7e1c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1565039321 shutdown_connections 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1565039321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2410f220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1565039321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1565039321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 0x7f7e24100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1565039321 >> 172.21.15.132:0/1565039321 conn(0x7f7e24077810 msgr2=0x7f7e240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:15.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1565039321 shutdown_connections 2024-03-18T23:08:15.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.486+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1565039321 wait complete. 2024-03-18T23:08:15.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 Processor -- start 2024-03-18T23:08:15.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 -- start start 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 0x7f7e2419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112740 con 0x7f7e241012c0 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241125c0 con 0x7f7e24100900 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2b4f2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241128c0 con 0x7f7e24107380 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e28a8d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e28a8d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40882/0 (socket says 172.21.15.132:40882) 2024-03-18T23:08:15.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e28a8d700 1 -- 172.21.15.132:0/1843548364 learned_addr learned my addr 172.21.15.132:0/1843548364 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e29a8f700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.487+0000 7f7e2928e700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 0x7f7e2419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 -- 172.21.15.132:0/1843548364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 msgr2=0x7f7e24198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 -- 172.21.15.132:0/1843548364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 msgr2=0x7f7e2419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 0x7f7e2419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 -- 172.21.15.132:0/1843548364 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241990c0 con 0x7f7e241012c0 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e29a8f700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e28a8d700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d980 tx=0x7f7e2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:15.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20016070 con 0x7f7e241012c0 2024-03-18T23:08:15.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e20004d10 con 0x7f7e241012c0 2024-03-18T23:08:15.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.488+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e241993b0 con 0x7f7e241012c0 2024-03-18T23:08:15.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.489+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20005670 con 0x7f7e241012c0 2024-03-18T23:08:15.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.489+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e24101a10 con 0x7f7e241012c0 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.490+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e24068fd0 con 0x7f7e241012c0 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.490+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f7e20004750 con 0x7f7e241012c0 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.491+0000 7f7e1a7fc700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 0x7f7e100a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.491+0000 7f7e2928e700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 0x7f7e100a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.491+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7e200ce7f0 con 0x7f7e241012c0 2024-03-18T23:08:15.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.492+0000 7f7e2928e700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 0x7f7e100a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e24199aa0 tx=0x7f7e14012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:15.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.496+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e2001f020 con 0x7f7e241012c0 2024-03-18T23:08:15.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 -- 172.21.15.132:0/1891499295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 msgr2=0x7f358410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.594 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 --2- 172.21.15.132:0/1891499295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f358410f220 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b0a0 tx=0x7f357c00b3b0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 -- 172.21.15.132:0/1891499295 shutdown_connections 2024-03-18T23:08:15.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 --2- 172.21.15.132:0/1891499295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584107380 0x7f3584107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 --2- 172.21.15.132:0/1891499295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f358410f220 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.595 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 --2- 172.21.15.132:0/1891499295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584100900 0x7f3584100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 -- 172.21.15.132:0/1891499295 >> 172.21.15.132:0/1891499295 conn(0x7f3584077810 msgr2=0x7f35840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:15.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 -- 172.21.15.132:0/1891499295 shutdown_connections 2024-03-18T23:08:15.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.661+0000 7f35890a4700 1 -- 172.21.15.132:0/1891499295 wait complete. 2024-03-18T23:08:15.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.662+0000 7f35890a4700 1 Processor -- start 2024-03-18T23:08:15.596 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.662+0000 7f35890a4700 1 -- start start 2024-03-18T23:08:15.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.662+0000 7f35890a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 0x7f3584101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f35890a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f35890a4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 0x7f3584102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f35890a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3584112760 con 0x7f35841012c0 2024-03-18T23:08:15.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f35890a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35841125e0 con 0x7f3584100900 2024-03-18T23:08:15.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f35890a4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35841128e0 con 0x7f3584107380 2024-03-18T23:08:15.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40826/0 (socket says 172.21.15.132:40826) 2024-03-18T23:08:15.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358359e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 0x7f3584102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 0x7f3584101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358259c700 1 -- 172.21.15.132:0/3232981619 learned_addr learned my addr 172.21.15.132:0/3232981619 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 -- 172.21.15.132:0/3232981619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 msgr2=0x7f3584102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 0x7f3584102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 -- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 msgr2=0x7f3584102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 -- 172.21.15.132:0/3232981619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35841a7110 con 0x7f3584100900 2024-03-18T23:08:15.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358359e700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 0x7f3584102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:15.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f358259c700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:15.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f3582d9d700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 0x7f3584101ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9f0 tx=0x7f357400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:15.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.663+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400c9c0 con 0x7f3584100900 2024-03-18T23:08:15.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.664+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35841a7400 con 0x7f3584100900 2024-03-18T23:08:15.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.664+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f357400cb60 con 0x7f3584100900 2024-03-18T23:08:15.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.664+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574018680 con 0x7f3584100900 2024-03-18T23:08:15.608 INFO:teuthology.orchestra.run.smithi132.stdout:266287972391 2024-03-18T23:08:15.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.664+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35841a7990 con 0x7f3584100900 2024-03-18T23:08:15.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.664+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3584068fd0 con 0x7f3584100900 2024-03-18T23:08:15.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.666+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3574010430 con 0x7f3584100900 2024-03-18T23:08:15.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.667+0000 7f356bfff700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 0x7f356c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.667+0000 7f358259c700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 0x7f356c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.667+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f35740cec90 con 0x7f3584100900 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.667+0000 7f358259c700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 0x7f356c0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3584106be0 tx=0x7f3578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.675+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f357409bd80 con 0x7f3584100900 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.880+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3584103c00 con 0x7f3584100900 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.881+0000 7f356bfff700 1 -- 172.21.15.132:0/3232981619 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f357409b680 con 0x7f3584100900 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 msgr2=0x7f356c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 0x7f356c0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3584106be0 tx=0x7f3578009260 comp rx=0 tx=0).stop 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 msgr2=0x7f3584101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 0x7f3584101ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f357400e9f0 tx=0x7f357400ed00 comp rx=0 tx=0).stop 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 shutdown_connections 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f356c0a2920 0x7f356c0a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.609 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3584100900 0x7f3584101ae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3584107380 0x7f3584102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 --2- 172.21.15.132:0/3232981619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35841012c0 0x7f3584102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 >> 172.21.15.132:0/3232981619 conn(0x7f3584077810 msgr2=0x7f3584105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:15.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 shutdown_connections 2024-03-18T23:08:15.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:13.883+0000 7f35890a4700 1 -- 172.21.15.132:0/3232981619 wait complete. 2024-03-18T23:08:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:15 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/780945869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T23:08:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:15 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:15 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:15 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:15 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:15.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.699+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7e24101c90 con 0x7f7e241012c0 2024-03-18T23:08:15.700 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.699+0000 7f7e1a7fc700 1 -- 172.21.15.132:0/1843548364 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7e20022350 con 0x7f7e241012c0 2024-03-18T23:08:15.700 INFO:teuthology.orchestra.run.smithi132.stdout:356482285594 2024-03-18T23:08:15.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 msgr2=0x7f7e100a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 0x7f7e100a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7e24199aa0 tx=0x7f7e14012040 comp rx=0 tx=0).stop 2024-03-18T23:08:15.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 msgr2=0x7f7e2419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:15.702 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d980 tx=0x7f7e2000dc90 comp rx=0 tx=0).stop 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 shutdown_connections 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f7e100a29a0 0x7f7e100a4e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e24100900 0x7f7e2419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24107380 0x7f7e24198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 --2- 172.21.15.132:0/1843548364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e241012c0 0x7f7e2419ece0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 >> 172.21.15.132:0/1843548364 conn(0x7f7e24077810 msgr2=0x7f7e24105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 shutdown_connections 2024-03-18T23:08:15.703 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:15.702+0000 7f7e2b4f2700 1 -- 172.21.15.132:0/1843548364 wait complete. 2024-03-18T23:08:16.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972388 got 266287972391 for osd.8 2024-03-18T23:08:16.249 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.433+0000 7f5a89a38700 1 -- 172.21.15.132:0/4130139746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a841014a0 msgr2=0x7f5a84101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.433+0000 7f5a89a38700 1 --2- 172.21.15.132:0/4130139746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a841014a0 0x7f5a84101880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009990 tx=0x7f5a74009ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 -- 172.21.15.132:0/4130139746 shutdown_connections 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 --2- 172.21.15.132:0/4130139746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 0x7f5a8410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 --2- 172.21.15.132:0/4130139746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a841014a0 0x7f5a84101880 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 --2- 172.21.15.132:0/4130139746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a84101e50 0x7f5a84105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 -- 172.21.15.132:0/4130139746 >> 172.21.15.132:0/4130139746 conn(0x7f5a84077220 msgr2=0x7f5a84077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 -- 172.21.15.132:0/4130139746 shutdown_connections 2024-03-18T23:08:16.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 -- 172.21.15.132:0/4130139746 wait complete. 2024-03-18T23:08:16.684 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:16 smithi132 ceph-mon[32020]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:16.684 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:16 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1843548364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:08:16.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 Processor -- start 2024-03-18T23:08:16.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.434+0000 7f5a89a38700 1 -- start start 2024-03-18T23:08:16.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 0x7f5a84103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 0x7f5a84103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5a841114e0 con 0x7f5a84101e50 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a84111360 con 0x7f5a84106520 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a89a38700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a84111660 con 0x7f5a841014a0 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a827fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 0x7f5a84103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:45186/0 (socket says 172.21.15.132:45186) 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 -- 172.21.15.132:0/3048522324 learned_addr learned my addr 172.21.15.132:0/3048522324 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a837fe700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 0x7f5a84103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 -- 172.21.15.132:0/3048522324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 msgr2=0x7f5a84103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 0x7f5a84103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 -- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 msgr2=0x7f5a84103420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 0x7f5a84103420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.687 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 -- 172.21.15.132:0/3048522324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a841a6fe0 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a827fc700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 0x7f5a84103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a82ffd700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a74000c00 tx=0x7f5a7400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74027070 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.435+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a74003d70 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.436+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a841a7270 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.436+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400f460 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.436+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a841a76c0 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.436+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a84102920 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.437+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f5a7402a070 con 0x7f5a841014a0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.438+0000 7f5a88a36700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 0x7f5a6c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.438+0000 7f5a827fc700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 0x7f5a6c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:16.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.438+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5a740cf770 con 0x7f5a841014a0 2024-03-18T23:08:16.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.438+0000 7f5a827fc700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 0x7f5a6c0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a84107c10 tx=0x7f5a78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:16.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.445+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a74013070 con 0x7f5a841014a0 2024-03-18T23:08:16.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.645+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5a8402dd70 con 0x7f5a841014a0 2024-03-18T23:08:16.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.645+0000 7f5a88a36700 1 -- 172.21.15.132:0/3048522324 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a7409c7f0 con 0x7f5a841014a0 2024-03-18T23:08:16.689 INFO:teuthology.orchestra.run.smithi132.stdout:111669149761 2024-03-18T23:08:16.690 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:16 smithi144 ceph-mon[66781]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:16.690 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:16 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1843548364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:08:16.690 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:16 smithi196 ceph-mon[67089]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:16.690 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:16 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1843548364' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 msgr2=0x7f5a6c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 0x7f5a6c0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a84107c10 tx=0x7f5a78009260 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 msgr2=0x7f5a84102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a74000c00 tx=0x7f5a7400bab0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 shutdown_connections 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5a6c0a2920 0x7f5a6c0a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a84106520 0x7f5a84103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a841014a0 0x7f5a84102ee0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 --2- 172.21.15.132:0/3048522324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a84101e50 0x7f5a84103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 >> 172.21.15.132:0/3048522324 conn(0x7f5a84077220 msgr2=0x7f5a8406ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:16.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 shutdown_connections 2024-03-18T23:08:16.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:16.648+0000 7f5a89a38700 1 -- 172.21.15.132:0/3048522324 wait complete. 2024-03-18T23:08:16.802 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411391 got 77309411397 for osd.2 2024-03-18T23:08:16.803 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:17.390 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:17 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3048522324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:08:17.414 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149761 for osd.3 2024-03-18T23:08:17.415 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:17.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:17 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3048522324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:08:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:17 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3048522324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T23:08:18.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:18 smithi132 ceph-mon[32020]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:18.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:18 smithi196 ceph-mon[67089]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:18 smithi144 ceph-mon[66781]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:19.318 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-18T23:08:19.318 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-18T23:08:19.318 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:08:19.319 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-18T23:08:19.319 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-18T23:08:19.319 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-18T23:08:19.319 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T23:08:19.319 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:19 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 -- 172.21.15.132:0/3947917789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b801083a0 msgr2=0x7f5b80108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 --2- 172.21.15.132:0/3947917789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b801083a0 0x7f5b80108780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009990 tx=0x7f5b78009ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 -- 172.21.15.132:0/3947917789 shutdown_connections 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 --2- 172.21.15.132:0/3947917789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b80102d30 0x7f5b8010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 --2- 172.21.15.132:0/3947917789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b801083a0 0x7f5b80108780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 --2- 172.21.15.132:0/3947917789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102390 0x7f5b801027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 -- 172.21.15.132:0/3947917789 >> 172.21.15.132:0/3947917789 conn(0x7f5b800793e0 msgr2=0x7f5b800797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 -- 172.21.15.132:0/3947917789 shutdown_connections 2024-03-18T23:08:19.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.328+0000 7f5b8844f700 1 -- 172.21.15.132:0/3947917789 wait complete. 2024-03-18T23:08:19.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 Processor -- start 2024-03-18T23:08:19.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 -- start start 2024-03-18T23:08:19.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102d30 0x7f5b8019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b801083a0 0x7f5b80198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80112600 con 0x7f5b80102390 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80112480 con 0x7f5b801083a0 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b8844f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80112780 con 0x7f5b80102d30 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:19.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40922/0 (socket says 172.21.15.132:40922) 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 -- 172.21.15.132:0/1862218470 learned_addr learned my addr 172.21.15.132:0/1862218470 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b869ec700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b801083a0 0x7f5b80198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b859ea700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102d30 0x7f5b8019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 -- 172.21.15.132:0/1862218470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102d30 msgr2=0x7f5b8019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102d30 0x7f5b8019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 -- 172.21.15.132:0/1862218470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b801083a0 msgr2=0x7f5b80198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b801083a0 0x7f5b80198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 -- 172.21.15.132:0/1862218470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b80199060 con 0x7f5b80102390 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.329+0000 7f5b861eb700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800b2e0 tx=0x7f5b7800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.330+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78026070 con 0x7f5b80102390 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.330+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801992f0 con 0x7f5b80102390 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.330+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7800fde0 con 0x7f5b80102390 2024-03-18T23:08:19.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.330+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780173f0 con 0x7f5b80102390 2024-03-18T23:08:19.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.330+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801a55c0 con 0x7f5b80102390 2024-03-18T23:08:19.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.331+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b80068fd0 con 0x7f5b80102390 2024-03-18T23:08:19.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.332+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f5b78029070 con 0x7f5b80102390 2024-03-18T23:08:19.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.332+0000 7f5b777fe700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 0x7f5b6c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:19.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.332+0000 7f5b859ea700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 0x7f5b6c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:19.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.332+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8211+0+0 (secure 0 0 0) 0x7f5b780cf8a0 con 0x7f5b80102390 2024-03-18T23:08:19.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.333+0000 7f5b859ea700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 0x7f5b6c0a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b80079a50 tx=0x7f5b70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:19.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.337+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b7805d740 con 0x7f5b80102390 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T23:08:19.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:19 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-18T23:08:19.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.535+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5b8002dbb0 con 0x7f5b80102390 2024-03-18T23:08:19.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.536+0000 7f5b777fe700 1 -- 172.21.15.132:0/1862218470 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b78013070 con 0x7f5b80102390 2024-03-18T23:08:19.537 INFO:teuthology.orchestra.run.smithi132.stdout:206158430258 2024-03-18T23:08:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 msgr2=0x7f5b6c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 0x7f5b6c0a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b80079a50 tx=0x7f5b70009260 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 msgr2=0x7f5b8019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800b2e0 tx=0x7f5b7800b610 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 shutdown_connections 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5b6c0a2920 0x7f5b6c0a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b801083a0 0x7f5b80198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b80102390 0x7f5b8019e6f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 --2- 172.21.15.132:0/1862218470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b80102d30 0x7f5b8019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 >> 172.21.15.132:0/1862218470 conn(0x7f5b800793e0 msgr2=0x7f5b80101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 shutdown_connections 2024-03-18T23:08:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:19.538+0000 7f5b8844f700 1 -- 172.21.15.132:0/1862218470 wait complete. 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T23:08:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:19 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-18T23:08:19.784 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285588 got 356482285594 for osd.11 2024-03-18T23:08:19.785 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:20.056 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:20.315 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:20.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:20 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1862218470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:20 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1862218470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-18T23:08:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-18T23:08:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T23:08:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-18T23:08:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T23:08:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:20 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1862218470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T23:08:21.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430252 got 206158430258 for osd.6 2024-03-18T23:08:21.202 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:21.424 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:21 smithi132 ceph-mon[32020]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:08:21.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 -- 172.21.15.132:0/4146697339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b44101e50 msgr2=0x7f1b44105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 --2- 172.21.15.132:0/4146697339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b44101e50 0x7f1b44105e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009920 tx=0x7f1b40009c30 comp rx=0 tx=0).stop 2024-03-18T23:08:21.449 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 -- 172.21.15.132:0/4146697339 shutdown_connections 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 --2- 172.21.15.132:0/4146697339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b441014a0 0x7f1b44101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 --2- 172.21.15.132:0/4146697339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441064f0 0x7f1b4410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 --2- 172.21.15.132:0/4146697339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b44101e50 0x7f1b44105e10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 -- 172.21.15.132:0/4146697339 >> 172.21.15.132:0/4146697339 conn(0x7f1b44077260 msgr2=0x7f1b44077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 -- 172.21.15.132:0/4146697339 shutdown_connections 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.446+0000 7f1b4c270700 1 -- 172.21.15.132:0/4146697339 wait complete. 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 Processor -- start 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 -- start start 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441014a0 0x7f1b4419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 0x7f1b4419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1b441114b0 con 0x7f1b441064f0 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44111330 con 0x7f1b44101e50 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4c270700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44111630 con 0x7f1b441014a0 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4a80d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4980b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 0x7f1b4419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4a80d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40938/0 (socket says 172.21.15.132:40938) 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4a80d700 1 -- 172.21.15.132:0/1317622862 learned_addr learned my addr 172.21.15.132:0/1317622862 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.447+0000 7f1b4a00c700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441014a0 0x7f1b4419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 -- 172.21.15.132:0/1317622862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441014a0 msgr2=0x7f1b4419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441014a0 0x7f1b4419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 -- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 msgr2=0x7f1b441989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 -- 172.21.15.132:0/1317622862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b44199120 con 0x7f1b44101e50 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4a80d700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4980b700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 0x7f1b4419ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000c00 tx=0x7f1b4000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40005040 con 0x7f1b44101e50 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b400056c0 con 0x7f1b44101e50 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400039b0 con 0x7f1b44101e50 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b441993b0 con 0x7f1b44101e50 2024-03-18T23:08:21.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.448+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a56d0 con 0x7f1b44101e50 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.449+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b44068fd0 con 0x7f1b44101e50 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.450+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f1b40003b90 con 0x7f1b44101e50 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.450+0000 7f1b3b7fe700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 0x7f1b300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.451+0000 7f1b4a00c700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 0x7f1b300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.451+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f1b400cfa40 con 0x7f1b44101e50 2024-03-18T23:08:21.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.451+0000 7f1b4a00c700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 0x7f1b300a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b44199b00 tx=0x7f1b34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:21.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.455+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b40013030 con 0x7f1b44101e50 2024-03-18T23:08:21.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:21 smithi196 ceph-mon[67089]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:08:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:21 smithi144 ceph-mon[66781]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T23:08:21.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 -- 172.21.15.132:0/2493404109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 msgr2=0x7f8a9410f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 --2- 172.21.15.132:0/2493404109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a9410f1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00b0d0 tx=0x7f8a8c00b3e0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 -- 172.21.15.132:0/2493404109 shutdown_connections 2024-03-18T23:08:21.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 --2- 172.21.15.132:0/2493404109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94108360 0x7f8a94108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 --2- 172.21.15.132:0/2493404109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a9410f1a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 --2- 172.21.15.132:0/2493404109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94069fc0 0x7f8a9406a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 -- 172.21.15.132:0/2493404109 >> 172.21.15.132:0/2493404109 conn(0x7f8a94078140 msgr2=0x7f8a94078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:21.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 -- 172.21.15.132:0/2493404109 shutdown_connections 2024-03-18T23:08:21.647 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.643+0000 7f8a990de700 1 -- 172.21.15.132:0/2493404109 wait complete. 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 Processor -- start 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 -- start start 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 0x7f8a94198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94108360 0x7f8a9419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.648 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941125b0 con 0x7f8a9406a960 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94112430 con 0x7f8a94069fc0 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a990de700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94112730 con 0x7f8a94108360 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51602/0 (socket says 172.21.15.132:51602) 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 -- 172.21.15.132:0/1309391262 learned_addr learned my addr 172.21.15.132:0/1309391262 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:21.649 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a92d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 0x7f8a94198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9359e700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94108360 0x7f8a9419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 -- 172.21.15.132:0/1309391262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94108360 msgr2=0x7f8a9419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94108360 0x7f8a9419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 -- 172.21.15.132:0/1309391262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 msgr2=0x7f8a94198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 0x7f8a94198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.650 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.644+0000 7f8a9259c700 1 -- 172.21.15.132:0/1309391262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9419daf0 con 0x7f8a9406a960 2024-03-18T23:08:21.651 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a92d9d700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 0x7f8a94198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:21.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a9259c700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800eb30 tx=0x7f8a8800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:21.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800cbd0 con 0x7f8a9406a960 2024-03-18T23:08:21.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9419dde0 con 0x7f8a9406a960 2024-03-18T23:08:21.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a88004500 con 0x7f8a9406a960 2024-03-18T23:08:21.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88010430 con 0x7f8a9406a960 2024-03-18T23:08:21.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.645+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a94101210 con 0x7f8a9406a960 2024-03-18T23:08:21.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.646+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94068fd0 con 0x7f8a9406a960 2024-03-18T23:08:21.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.648+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8a88003f00 con 0x7f8a9406a960 2024-03-18T23:08:21.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.649+0000 7f8a7bfff700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 0x7f8a7c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:21.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.649+0000 7f8a92d9d700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 0x7f8a7c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:21.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.649+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f8a880cef40 con 0x7f8a9406a960 2024-03-18T23:08:21.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.649+0000 7f8a92d9d700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 0x7f8a7c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009fd0 tx=0x7f8a84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:21.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.651+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a8801f340 con 0x7f8a9406a960 2024-03-18T23:08:21.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.669+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1b4402dbb0 con 0x7f1b44101e50 2024-03-18T23:08:21.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.669+0000 7f1b3b7fe700 1 -- 172.21.15.132:0/1317622862 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1b4009ca20 con 0x7f1b44101e50 2024-03-18T23:08:21.671 INFO:teuthology.orchestra.run.smithi132.stdout:326417514528 2024-03-18T23:08:21.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 msgr2=0x7f1b300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 0x7f1b300a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b44199b00 tx=0x7f1b34009310 comp rx=0 tx=0).stop 2024-03-18T23:08:21.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 msgr2=0x7f1b4419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 0x7f1b4419ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000c00 tx=0x7f1b4000fa00 comp rx=0 tx=0).stop 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 shutdown_connections 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1b300a29a0 0x7f1b300a4e60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b441064f0 0x7f1b441989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b441014a0 0x7f1b4419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 --2- 172.21.15.132:0/1317622862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b44101e50 0x7f1b4419ed40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.675 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 >> 172.21.15.132:0/1317622862 conn(0x7f1b44077260 msgr2=0x7f1b4406abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:21.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 shutdown_connections 2024-03-18T23:08:21.676 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.673+0000 7f1b4c270700 1 -- 172.21.15.132:0/1317622862 wait complete. 2024-03-18T23:08:21.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.860+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8a94199e50 con 0x7f8a9406a960 2024-03-18T23:08:21.861 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.860+0000 7f8a7bfff700 1 -- 172.21.15.132:0/1309391262 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a8809bf20 con 0x7f8a9406a960 2024-03-18T23:08:21.863 INFO:teuthology.orchestra.run.smithi132.stdout:55834574924 2024-03-18T23:08:21.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.862+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 msgr2=0x7f8a7c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.862+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 0x7f8a7c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009fd0 tx=0x7f8a84009260 comp rx=0 tx=0).stop 2024-03-18T23:08:21.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.862+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 msgr2=0x7f8a94199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.862+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800eb30 tx=0x7f8a8800ee40 comp rx=0 tx=0).stop 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 shutdown_connections 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a7c0a29a0 0x7f8a7c0a4e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a94069fc0 0x7f8a94198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a94108360 0x7f8a9419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 --2- 172.21.15.132:0/1309391262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9406a960 0x7f8a94199080 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 >> 172.21.15.132:0/1309391262 conn(0x7f8a94078140 msgr2=0x7f8a940ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 shutdown_connections 2024-03-18T23:08:21.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:21.863+0000 7f8a990de700 1 -- 172.21.15.132:0/1309391262 wait complete. 2024-03-18T23:08:22.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:22 smithi196 ceph-mon[67089]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-18T23:08:22.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:22 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1317622862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:08:22.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:22 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1309391262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:08:22.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:22 smithi132 ceph-mon[32020]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-18T23:08:22.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:22 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1317622862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:08:22.504 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:22 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1309391262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:08:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:22 smithi144 ceph-mon[66781]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-18T23:08:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:22 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1317622862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T23:08:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:22 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1309391262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T23:08:23.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574924 for osd.1 2024-03-18T23:08:23.076 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:23.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:23 smithi132 ceph-mon[32020]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:23.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:23 smithi196 ceph-mon[67089]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:23 smithi144 ceph-mon[66781]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:24.102 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 -- 172.21.15.132:0/1252642946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 msgr2=0x7f5cf010e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/1252642946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf010e090 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009f90 tx=0x7f5ce800b330 comp rx=0 tx=0).stop 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 -- 172.21.15.132:0/1252642946 shutdown_connections 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/1252642946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 0x7f5cf0107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/1252642946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf010e090 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/1252642946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 0x7f5cf0101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 -- 172.21.15.132:0/1252642946 >> 172.21.15.132:0/1252642946 conn(0x7f5cf0078140 msgr2=0x7f5cf0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 -- 172.21.15.132:0/1252642946 shutdown_connections 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.095+0000 7f5cf53f7700 1 -- 172.21.15.132:0/1252642946 wait complete. 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 Processor -- start 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 -- start start 2024-03-18T23:08:24.103 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 0x7f5cf01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 0x7f5cf019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01114a0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0111320 con 0x7f5cf0107250 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cf53f7700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0111620 con 0x7f5cf0101210 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51614/0 (socket says 172.21.15.132:51614) 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 -- 172.21.15.132:0/3212650718 learned_addr learned my addr 172.21.15.132:0/3212650718 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5cef7fe700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 0x7f5cf019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ceeffd700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 0x7f5cf01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 -- 172.21.15.132:0/3212650718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 msgr2=0x7f5cf01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 0x7f5cf01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.104 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 -- 172.21.15.132:0/3212650718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 msgr2=0x7f5cf019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 0x7f5cf019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.096+0000 7f5ce67fc700 1 -- 172.21.15.132:0/3212650718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf01a4d90 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5ce67fc700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9c0 tx=0x7f5cd800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004a30 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01a5080 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd800cca0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8005610 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.097+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a55c0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.098+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf0068fd0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.099+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f5cd8004cf0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.099+0000 7f5cecff9700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 0x7f5cdc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.099+0000 7f5ceeffd700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 0x7f5cdc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.105 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.099+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5cd80ceed0 con 0x7f5cf0101bb0 2024-03-18T23:08:24.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.100+0000 7f5ceeffd700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 0x7f5cdc0a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009be0 tx=0x7f5ce0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:24.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.103+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5cd8053740 con 0x7f5cf0101bb0 2024-03-18T23:08:24.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.305+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5cf006a560 con 0x7f5cf0101bb0 2024-03-18T23:08:24.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.306+0000 7f5cecff9700 1 -- 172.21.15.132:0/3212650718 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5cd809be20 con 0x7f5cf0101bb0 2024-03-18T23:08:24.306 INFO:teuthology.orchestra.run.smithi132.stdout:34359738447 2024-03-18T23:08:24.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 msgr2=0x7f5cdc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 0x7f5cdc0a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009be0 tx=0x7f5ce0009260 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 msgr2=0x7f5cf0199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9c0 tx=0x7f5cd800ecd0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 shutdown_connections 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f5cdc0a2920 0x7f5cdc0a4de0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cf0107250 0x7f5cf019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf0101210 0x7f5cf01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 --2- 172.21.15.132:0/3212650718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0101bb0 0x7f5cf0199ee0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 >> 172.21.15.132:0/3212650718 conn(0x7f5cf0078140 msgr2=0x7f5cf00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 shutdown_connections 2024-03-18T23:08:24.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.308+0000 7f5cf53f7700 1 -- 172.21.15.132:0/3212650718 wait complete. 2024-03-18T23:08:24.498 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514523 got 326417514528 for osd.10 2024-03-18T23:08:24.498 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:24 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3212650718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:08:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:24 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3212650718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:08:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:24 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3212650718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T23:08:24.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 -- 172.21.15.132:0/2333598696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a40101e90 msgr2=0x7f8a40105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2333598696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a40101e90 0x7f8a40105ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a300099c0 tx=0x7f8a30009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 -- 172.21.15.132:0/2333598696 shutdown_connections 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2333598696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a401014e0 0x7f8a401018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2333598696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 0x7f8a4010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2333598696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a40101e90 0x7f8a40105ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 -- 172.21.15.132:0/2333598696 >> 172.21.15.132:0/2333598696 conn(0x7f8a40078340 msgr2=0x7f8a40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 -- 172.21.15.132:0/2333598696 shutdown_connections 2024-03-18T23:08:24.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 -- 172.21.15.132:0/2333598696 wait complete. 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.776+0000 7f8a47d63700 1 Processor -- start 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- start start 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a40101e90 0x7f8a40199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 0x7f8a401a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a401115a0 con 0x7f8a401014e0 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40111420 con 0x7f8a40101e90 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40111720 con 0x7f8a401065c0 2024-03-18T23:08:24.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51634/0 (socket says 172.21.15.132:51634) 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 -- 172.21.15.132:0/2691507521 learned_addr learned my addr 172.21.15.132:0/2691507521 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a46300700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 0x7f8a401a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a452fe700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a40101e90 0x7f8a40199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 -- 172.21.15.132:0/2691507521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 msgr2=0x7f8a401a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 0x7f8a401a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 -- 172.21.15.132:0/2691507521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a40101e90 msgr2=0x7f8a40199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a40101e90 0x7f8a40199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 -- 172.21.15.132:0/2691507521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a401a4ea0 con 0x7f8a401014e0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a45aff700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9a0 tx=0x7f8a3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00ca10 con 0x7f8a401014e0 2024-03-18T23:08:24.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a401a5130 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a3c00cbb0 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c018680 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.777+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a401a5640 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.778+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a40068fd0 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.779+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8a3c010430 con 0x7f8a401014e0 2024-03-18T23:08:24.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.780+0000 7f8a36ffd700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 0x7f8a2c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:24.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.780+0000 7f8a452fe700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 0x7f8a2c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:24.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.780+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f8a3c0cf290 con 0x7f8a401014e0 2024-03-18T23:08:24.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.780+0000 7f8a452fe700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 0x7f8a2c0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000c00 tx=0x7f8a30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:24.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.784+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a3c09c1e0 con 0x7f8a401014e0 2024-03-18T23:08:24.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.987+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8a4002dbb0 con 0x7f8a401014e0 2024-03-18T23:08:24.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.987+0000 7f8a36ffd700 1 -- 172.21.15.132:0/2691507521 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a3c09bae0 con 0x7f8a401014e0 2024-03-18T23:08:24.988 INFO:teuthology.orchestra.run.smithi132.stdout:300647710758 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 msgr2=0x7f8a2c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 0x7f8a2c0a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000c00 tx=0x7f8a30005fb0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 msgr2=0x7f8a401998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9a0 tx=0x7f8a3c00ecb0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 shutdown_connections 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f8a2c0a2920 0x7f8a2c0a4de0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a401065c0 0x7f8a401a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a401014e0 0x7f8a401998e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 --2- 172.21.15.132:0/2691507521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a40101e90 0x7f8a40199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:24.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 >> 172.21.15.132:0/2691507521 conn(0x7f8a40078340 msgr2=0x7f8a400ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:24.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 shutdown_connections 2024-03-18T23:08:24.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:24.989+0000 7f8a47d63700 1 -- 172.21.15.132:0/2691507521 wait complete. 2024-03-18T23:08:25.531 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:25 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2691507521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:08:25.532 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:25 smithi132 ceph-mon[32020]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:25 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2691507521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:08:25.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:25 smithi196 ceph-mon[67089]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:25.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738447 for osd.0 2024-03-18T23:08:25.794 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:25 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2691507521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T23:08:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:25 smithi144 ceph-mon[66781]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:26.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710753 got 300647710758 for osd.9 2024-03-18T23:08:26.296 DEBUG:teuthology.parallel:result is None 2024-03-18T23:08:26.296 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-18T23:08:26.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T23:08:26.604 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 -- 172.21.15.132:0/3130206127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c9406a960 msgr2=0x7f0c9410f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3130206127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c9406a960 0x7f0c9410f1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b0d0 tx=0x7f0c8c00b3e0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 -- 172.21.15.132:0/3130206127 shutdown_connections 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3130206127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94108360 0x7f0c94108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3130206127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c9406a960 0x7f0c9410f1a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3130206127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c94069fc0 0x7f0c9406a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 -- 172.21.15.132:0/3130206127 >> 172.21.15.132:0/3130206127 conn(0x7f0c94078140 msgr2=0x7f0c94078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:28.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 -- 172.21.15.132:0/3130206127 shutdown_connections 2024-03-18T23:08:28.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.223+0000 7f0c9b123700 1 -- 172.21.15.132:0/3130206127 wait complete. 2024-03-18T23:08:28.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 Processor -- start 2024-03-18T23:08:28.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 -- start start 2024-03-18T23:08:28.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:28.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 0x7f0c9419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:28.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 0x7f0c941988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:28.226 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941125e0 con 0x7f0c94108360 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94112460 con 0x7f0c94069fc0 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c9b123700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94112760 con 0x7f0c9406a960 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c98ebf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c93fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 0x7f0c9419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c98ebf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:35786/0 (socket says 172.21.15.132:35786) 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 0x7f0c941988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c98ebf700 1 -- 172.21.15.132:0/3329088393 learned_addr learned my addr 172.21.15.132:0/3329088393 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 -- 172.21.15.132:0/3329088393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 msgr2=0x7f0c9419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 0x7f0c9419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 -- 172.21.15.132:0/3329088393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 msgr2=0x7f0c9419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c996c0700 1 -- 172.21.15.132:0/3329088393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c94199080 con 0x7f0c94108360 2024-03-18T23:08:28.227 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c93fff700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 0x7f0c9419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.224+0000 7f0c98ebf700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c996c0700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 0x7f0c941988a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b770 tx=0x7f0c8c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c021020 con 0x7f0c94108360 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c8c004c50 con 0x7f0c94108360 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c94199310 con 0x7f0c94108360 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0052b0 con 0x7f0c94108360 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.225+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c941a5530 con 0x7f0c94108360 2024-03-18T23:08:28.228 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.226+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c94103a10 con 0x7f0c94108360 2024-03-18T23:08:28.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.227+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f0c8c024020 con 0x7f0c94108360 2024-03-18T23:08:28.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.227+0000 7f0c91ffb700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 0x7f0c840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:28.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.228+0000 7f0c98ebf700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 0x7f0c840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:28.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.228+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0c8c00f020 con 0x7f0c94108360 2024-03-18T23:08:28.229 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.228+0000 7f0c98ebf700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 0x7f0c840a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c88011550 tx=0x7f0c88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:28.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.232+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0c8c09caa0 con 0x7f0c94108360 2024-03-18T23:08:28.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:28 smithi132 ceph-mon[32020]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:28.440 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.439+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c94062750 con 0x7f0c840a2920 2024-03-18T23:08:28.447 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.446+0000 7f0c91ffb700 1 -- 172.21.15.132:0/3329088393 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258707 (secure 0 0 0) 0x7f0c94062750 con 0x7f0c840a2920 2024-03-18T23:08:28.451 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:28.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.450+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 msgr2=0x7f0c840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.450+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 0x7f0c840a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c88011550 tx=0x7f0c88009260 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 msgr2=0x7f0c941988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 0x7f0c941988a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b770 tx=0x7f0c8c007a20 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 shutdown_connections 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0c840a2920 0x7f0c840a4de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c94069fc0 0x7f0c9419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c94108360 0x7f0c941988a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 --2- 172.21.15.132:0/3329088393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c9406a960 0x7f0c9419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 >> 172.21.15.132:0/3329088393 conn(0x7f0c94078140 msgr2=0x7f0c940ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 shutdown_connections 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:28.451+0000 7f0c9b123700 1 -- 172.21.15.132:0/3329088393 wait complete. 2024-03-18T23:08:28.453 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-18T23:08:28.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:28 smithi196 ceph-mon[67089]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:28.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:28 smithi144 ceph-mon[66781]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 5 op/s; 75 B/s, 0 objects/s recovering 2024-03-18T23:08:29.153 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-03-18T23:08:27.061908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653906,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":558,"num_read_kb":399,"num_write":338,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181340,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371092,"kb_used_data":13548,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570732,"statfs":{"total":1151940427776,"available":1151560429568,"internally_reserved":0,"allocated":13873152,"data_stored":7581162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001858"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195321+0000","last_change":"2024-03-18T23:07:14.066593+0000","last_active":"2024-03-18T23:08:20.195321+0000","last_peered":"2024-03-18T23:08:20.195321+0000","last_clean":"2024-03-18T23:08:20.195321+0000","last_became_active":"2024-03-18T23:07:14.066064+0000","last_became_peered":"2024-03-18T23:07:14.066064+0000","last_unstale":"2024-03-18T23:08:20.195321+0000","last_undegraded":"2024-03-18T23:08:20.195321+0000","last_fullsized":"2024-03-18T23:08:20.195321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:48.122520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196884+0000","last_change":"2024-03-18T23:07:14.070484+0000","last_active":"2024-03-18T23:08:20.196884+0000","last_peered":"2024-03-18T23:08:20.196884+0000","last_clean":"2024-03-18T23:08:20.196884+0000","last_became_active":"2024-03-18T23:07:14.070129+0000","last_became_peered":"2024-03-18T23:07:14.070129+0000","last_unstale":"2024-03-18T23:08:20.196884+0000","last_undegraded":"2024-03-18T23:08:20.196884+0000","last_fullsized":"2024-03-18T23:08:20.196884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:13:44.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196727+0000","last_change":"2024-03-18T23:07:14.064923+0000","last_active":"2024-03-18T23:08:20.196727+0000","last_peered":"2024-03-18T23:08:20.196727+0000","last_clean":"2024-03-18T23:08:20.196727+0000","last_became_active":"2024-03-18T23:07:14.064513+0000","last_became_peered":"2024-03-18T23:07:14.064513+0000","last_unstale":"2024-03-18T23:08:20.196727+0000","last_undegraded":"2024-03-18T23:08:20.196727+0000","last_fullsized":"2024-03-18T23:08:20.196727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:33.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.200142+0000","last_change":"2024-03-18T23:08:20.199613+0000","last_active":"2024-03-18T23:08:20.200142+0000","last_peered":"2024-03-18T23:08:20.200142+0000","last_clean":"2024-03-18T23:08:20.200142+0000","last_became_active":"2024-03-18T23:08:20.199160+0000","last_became_peered":"2024-03-18T23:08:20.199160+0000","last_unstale":"2024-03-18T23:08:20.200142+0000","last_undegraded":"2024-03-18T23:08:20.200142+0000","last_fullsized":"2024-03-18T23:08:20.200142+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:20.345400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197015+0000","last_change":"2024-03-18T23:07:14.058801+0000","last_active":"2024-03-18T23:08:20.197015+0000","last_peered":"2024-03-18T23:08:20.197015+0000","last_clean":"2024-03-18T23:08:20.197015+0000","last_became_active":"2024-03-18T23:07:14.058481+0000","last_became_peered":"2024-03-18T23:07:14.058481+0000","last_unstale":"2024-03-18T23:08:20.197015+0000","last_undegraded":"2024-03-18T23:08:20.197015+0000","last_fullsized":"2024-03-18T23:08:20.197015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:55.420890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813500+0000","last_change":"2024-03-18T23:07:14.067320+0000","last_active":"2024-03-18T23:08:20.813500+0000","last_peered":"2024-03-18T23:08:20.813500+0000","last_clean":"2024-03-18T23:08:20.813500+0000","last_became_active":"2024-03-18T23:07:14.066987+0000","last_became_peered":"2024-03-18T23:07:14.066987+0000","last_unstale":"2024-03-18T23:08:20.813500+0000","last_undegraded":"2024-03-18T23:08:20.813500+0000","last_fullsized":"2024-03-18T23:08:20.813500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:52:33.006026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156112+0000","last_change":"2024-03-18T23:07:14.065320+0000","last_active":"2024-03-18T23:08:21.156112+0000","last_peered":"2024-03-18T23:08:21.156112+0000","last_clean":"2024-03-18T23:08:21.156112+0000","last_became_active":"2024-03-18T23:07:14.064280+0000","last_became_peered":"2024-03-18T23:07:14.064280+0000","last_unstale":"2024-03-18T23:08:21.156112+0000","last_undegraded":"2024-03-18T23:08:21.156112+0000","last_fullsized":"2024-03-18T23:08:21.156112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:07.838038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.815904+0000","last_change":"2024-03-18T23:08:20.815124+0000","last_active":"2024-03-18T23:08:20.815904+0000","last_peered":"2024-03-18T23:08:20.815904+0000","last_clean":"2024-03-18T23:08:20.815904+0000","last_became_active":"2024-03-18T23:08:20.814510+0000","last_became_peered":"2024-03-18T23:08:20.814510+0000","last_unstale":"2024-03-18T23:08:20.815904+0000","last_undegraded":"2024-03-18T23:08:20.815904+0000","last_fullsized":"2024-03-18T23:08:20.815904+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:18.592085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191604+0000","last_change":"2024-03-18T23:07:14.072829+0000","last_active":"2024-03-18T23:08:20.191604+0000","last_peered":"2024-03-18T23:08:20.191604+0000","last_clean":"2024-03-18T23:08:20.191604+0000","last_became_active":"2024-03-18T23:07:14.072493+0000","last_became_peered":"2024-03-18T23:07:14.072493+0000","last_unstale":"2024-03-18T23:08:20.191604+0000","last_undegraded":"2024-03-18T23:08:20.191604+0000","last_fullsized":"2024-03-18T23:08:20.191604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:58:12.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813792+0000","last_change":"2024-03-18T23:07:14.067119+0000","last_active":"2024-03-18T23:08:20.813792+0000","last_peered":"2024-03-18T23:08:20.813792+0000","last_clean":"2024-03-18T23:08:20.813792+0000","last_became_active":"2024-03-18T23:07:14.066738+0000","last_became_peered":"2024-03-18T23:07:14.066738+0000","last_unstale":"2024-03-18T23:08:20.813792+0000","last_undegraded":"2024-03-18T23:08:20.813792+0000","last_fullsized":"2024-03-18T23:08:20.813792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:55.011692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156513+0000","last_change":"2024-03-18T23:07:14.071791+0000","last_active":"2024-03-18T23:08:21.156513+0000","last_peered":"2024-03-18T23:08:21.156513+0000","last_clean":"2024-03-18T23:08:21.156513+0000","last_became_active":"2024-03-18T23:07:14.068031+0000","last_became_peered":"2024-03-18T23:07:14.068031+0000","last_unstale":"2024-03-18T23:08:21.156513+0000","last_undegraded":"2024-03-18T23:08:21.156513+0000","last_fullsized":"2024-03-18T23:08:21.156513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:42:56.142512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196919+0000","last_change":"2024-03-18T23:07:14.071712+0000","last_active":"2024-03-18T23:08:20.196919+0000","last_peered":"2024-03-18T23:08:20.196919+0000","last_clean":"2024-03-18T23:08:20.196919+0000","last_became_active":"2024-03-18T23:07:14.069595+0000","last_became_peered":"2024-03-18T23:07:14.069595+0000","last_unstale":"2024-03-18T23:08:20.196919+0000","last_undegraded":"2024-03-18T23:08:20.196919+0000","last_fullsized":"2024-03-18T23:08:20.196919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:10:21.521681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.815159+0000","last_change":"2024-03-18T23:08:20.815159+0000","last_active":"2024-03-18T23:08:20.815159+0000","last_peered":"2024-03-18T23:08:20.815159+0000","last_clean":"2024-03-18T23:08:20.815159+0000","last_became_active":"2024-03-18T23:08:20.814675+0000","last_became_peered":"2024-03-18T23:08:20.814675+0000","last_unstale":"2024-03-18T23:08:20.815159+0000","last_undegraded":"2024-03-18T23:08:20.815159+0000","last_fullsized":"2024-03-18T23:08:20.815159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:08:36.453847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196696+0000","last_change":"2024-03-18T23:07:14.070755+0000","last_active":"2024-03-18T23:08:20.196696+0000","last_peered":"2024-03-18T23:08:20.196696+0000","last_clean":"2024-03-18T23:08:20.196696+0000","last_became_active":"2024-03-18T23:07:14.070126+0000","last_became_peered":"2024-03-18T23:07:14.070126+0000","last_unstale":"2024-03-18T23:08:20.196696+0000","last_undegraded":"2024-03-18T23:08:20.196696+0000","last_fullsized":"2024-03-18T23:08:20.196696+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:05.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196553+0000","last_change":"2024-03-18T23:07:14.070166+0000","last_active":"2024-03-18T23:08:20.196553+0000","last_peered":"2024-03-18T23:08:20.196553+0000","last_clean":"2024-03-18T23:08:20.196553+0000","last_became_active":"2024-03-18T23:07:14.069693+0000","last_became_peered":"2024-03-18T23:07:14.069693+0000","last_unstale":"2024-03-18T23:08:20.196553+0000","last_undegraded":"2024-03-18T23:08:20.196553+0000","last_fullsized":"2024-03-18T23:08:20.196553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:55.616894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191091+0000","last_change":"2024-03-18T23:07:14.076732+0000","last_active":"2024-03-18T23:08:20.191091+0000","last_peered":"2024-03-18T23:08:20.191091+0000","last_clean":"2024-03-18T23:08:20.191091+0000","last_became_active":"2024-03-18T23:07:14.076143+0000","last_became_peered":"2024-03-18T23:07:14.076143+0000","last_unstale":"2024-03-18T23:08:20.191091+0000","last_undegraded":"2024-03-18T23:08:20.191091+0000","last_fullsized":"2024-03-18T23:08:20.191091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:46.888524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195486+0000","last_change":"2024-03-18T23:07:14.066388+0000","last_active":"2024-03-18T23:08:20.195486+0000","last_peered":"2024-03-18T23:08:20.195486+0000","last_clean":"2024-03-18T23:08:20.195486+0000","last_became_active":"2024-03-18T23:07:14.065626+0000","last_became_peered":"2024-03-18T23:07:14.065626+0000","last_unstale":"2024-03-18T23:08:20.195486+0000","last_undegraded":"2024-03-18T23:08:20.195486+0000","last_fullsized":"2024-03-18T23:08:20.195486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:08:18.200855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193357+0000","last_change":"2024-03-18T23:07:14.068730+0000","last_active":"2024-03-18T23:08:20.193357+0000","last_peered":"2024-03-18T23:08:20.193357+0000","last_clean":"2024-03-18T23:08:20.193357+0000","last_became_active":"2024-03-18T23:07:14.068420+0000","last_became_peered":"2024-03-18T23:07:14.068420+0000","last_unstale":"2024-03-18T23:08:20.193357+0000","last_undegraded":"2024-03-18T23:08:20.193357+0000","last_fullsized":"2024-03-18T23:08:20.193357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:41:34.588823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198509+0000","last_change":"2024-03-18T23:07:14.061415+0000","last_active":"2024-03-18T23:08:20.198509+0000","last_peered":"2024-03-18T23:08:20.198509+0000","last_clean":"2024-03-18T23:08:20.198509+0000","last_became_active":"2024-03-18T23:07:14.061102+0000","last_became_peered":"2024-03-18T23:07:14.061102+0000","last_unstale":"2024-03-18T23:08:20.198509+0000","last_undegraded":"2024-03-18T23:08:20.198509+0000","last_fullsized":"2024-03-18T23:08:20.198509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:06:08.056213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.192985+0000","last_change":"2024-03-18T23:07:13.506404+0000","last_active":"2024-03-18T23:08:20.192985+0000","last_peered":"2024-03-18T23:08:20.192985+0000","last_clean":"2024-03-18T23:08:20.192985+0000","last_became_active":"2024-03-18T23:07:13.506091+0000","last_became_peered":"2024-03-18T23:07:13.506091+0000","last_unstale":"2024-03-18T23:08:20.192985+0000","last_undegraded":"2024-03-18T23:08:20.192985+0000","last_fullsized":"2024-03-18T23:08:20.192985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:34:26.916813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914771+0000","last_change":"2024-03-18T23:07:14.061990+0000","last_active":"2024-03-18T23:08:20.914771+0000","last_peered":"2024-03-18T23:08:20.914771+0000","last_clean":"2024-03-18T23:08:20.914771+0000","last_became_active":"2024-03-18T23:07:14.061581+0000","last_became_peered":"2024-03-18T23:07:14.061581+0000","last_unstale":"2024-03-18T23:08:20.914771+0000","last_undegraded":"2024-03-18T23:08:20.914771+0000","last_fullsized":"2024-03-18T23:08:20.914771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:37.726342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914806+0000","last_change":"2024-03-18T23:07:13.509727+0000","last_active":"2024-03-18T23:08:20.914806+0000","last_peered":"2024-03-18T23:08:20.914806+0000","last_clean":"2024-03-18T23:08:20.914806+0000","last_became_active":"2024-03-18T23:07:13.509296+0000","last_became_peered":"2024-03-18T23:07:13.509296+0000","last_unstale":"2024-03-18T23:08:20.914806+0000","last_undegraded":"2024-03-18T23:08:20.914806+0000","last_fullsized":"2024-03-18T23:08:20.914806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:15.060387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197181+0000","last_change":"2024-03-18T23:07:14.066038+0000","last_active":"2024-03-18T23:08:20.197181+0000","last_peered":"2024-03-18T23:08:20.197181+0000","last_clean":"2024-03-18T23:08:20.197181+0000","last_became_active":"2024-03-18T23:07:14.065449+0000","last_became_peered":"2024-03-18T23:07:14.065449+0000","last_unstale":"2024-03-18T23:08:20.197181+0000","last_undegraded":"2024-03-18T23:08:20.197181+0000","last_fullsized":"2024-03-18T23:08:20.197181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:53:07.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197350+0000","last_change":"2024-03-18T23:07:13.503422+0000","last_active":"2024-03-18T23:08:20.197350+0000","last_peered":"2024-03-18T23:08:20.197350+0000","last_clean":"2024-03-18T23:08:20.197350+0000","last_became_active":"2024-03-18T23:07:13.503068+0000","last_became_peered":"2024-03-18T23:07:13.503068+0000","last_unstale":"2024-03-18T23:08:20.197350+0000","last_undegraded":"2024-03-18T23:08:20.197350+0000","last_fullsized":"2024-03-18T23:08:20.197350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:35.369317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191189+0000","last_change":"2024-03-18T23:07:14.073092+0000","last_active":"2024-03-18T23:08:20.191189+0000","last_peered":"2024-03-18T23:08:20.191189+0000","last_clean":"2024-03-18T23:08:20.191189+0000","last_became_active":"2024-03-18T23:07:14.070303+0000","last_became_peered":"2024-03-18T23:07:14.070303+0000","last_unstale":"2024-03-18T23:08:20.191189+0000","last_undegraded":"2024-03-18T23:08:20.191189+0000","last_fullsized":"2024-03-18T23:08:20.191189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:09.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197336+0000","last_change":"2024-03-18T23:07:13.505420+0000","last_active":"2024-03-18T23:08:20.197336+0000","last_peered":"2024-03-18T23:08:20.197336+0000","last_clean":"2024-03-18T23:08:20.197336+0000","last_became_active":"2024-03-18T23:07:13.504995+0000","last_became_peered":"2024-03-18T23:07:13.504995+0000","last_unstale":"2024-03-18T23:08:20.197336+0000","last_undegraded":"2024-03-18T23:08:20.197336+0000","last_fullsized":"2024-03-18T23:08:20.197336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:50.988318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191481+0000","last_change":"2024-03-18T23:07:14.071254+0000","last_active":"2024-03-18T23:08:20.191481+0000","last_peered":"2024-03-18T23:08:20.191481+0000","last_clean":"2024-03-18T23:08:20.191481+0000","last_became_active":"2024-03-18T23:07:14.070767+0000","last_became_peered":"2024-03-18T23:07:14.070767+0000","last_unstale":"2024-03-18T23:08:20.191481+0000","last_undegraded":"2024-03-18T23:08:20.191481+0000","last_fullsized":"2024-03-18T23:08:20.191481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:36.639093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.814603+0000","last_change":"2024-03-18T23:07:13.505020+0000","last_active":"2024-03-18T23:08:20.814603+0000","last_peered":"2024-03-18T23:08:20.814603+0000","last_clean":"2024-03-18T23:08:20.814603+0000","last_became_active":"2024-03-18T23:07:13.504355+0000","last_became_peered":"2024-03-18T23:07:13.504355+0000","last_unstale":"2024-03-18T23:08:20.814603+0000","last_undegraded":"2024-03-18T23:08:20.814603+0000","last_fullsized":"2024-03-18T23:08:20.814603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:03.184513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196979+0000","last_change":"2024-03-18T23:07:14.063156+0000","last_active":"2024-03-18T23:08:20.196979+0000","last_peered":"2024-03-18T23:08:20.196979+0000","last_clean":"2024-03-18T23:08:20.196979+0000","last_became_active":"2024-03-18T23:07:14.062801+0000","last_became_peered":"2024-03-18T23:07:14.062801+0000","last_unstale":"2024-03-18T23:08:20.196979+0000","last_undegraded":"2024-03-18T23:08:20.196979+0000","last_fullsized":"2024-03-18T23:08:20.196979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:49:27.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813469+0000","last_change":"2024-03-18T23:07:13.501538+0000","last_active":"2024-03-18T23:08:20.813469+0000","last_peered":"2024-03-18T23:08:20.813469+0000","last_clean":"2024-03-18T23:08:20.813469+0000","last_became_active":"2024-03-18T23:07:13.501160+0000","last_became_peered":"2024-03-18T23:07:13.501160+0000","last_unstale":"2024-03-18T23:08:20.813469+0000","last_undegraded":"2024-03-18T23:08:20.813469+0000","last_fullsized":"2024-03-18T23:08:20.813469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.319636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191446+0000","last_change":"2024-03-18T23:07:14.069345+0000","last_active":"2024-03-18T23:08:20.191446+0000","last_peered":"2024-03-18T23:08:20.191446+0000","last_clean":"2024-03-18T23:08:20.191446+0000","last_became_active":"2024-03-18T23:07:14.068320+0000","last_became_peered":"2024-03-18T23:07:14.068320+0000","last_unstale":"2024-03-18T23:08:20.191446+0000","last_undegraded":"2024-03-18T23:08:20.191446+0000","last_fullsized":"2024-03-18T23:08:20.191446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:36.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193761+0000","last_change":"2024-03-18T23:07:13.507538+0000","last_active":"2024-03-18T23:08:20.193761+0000","last_peered":"2024-03-18T23:08:20.193761+0000","last_clean":"2024-03-18T23:08:20.193761+0000","last_became_active":"2024-03-18T23:07:13.506563+0000","last_became_peered":"2024-03-18T23:07:13.506563+0000","last_unstale":"2024-03-18T23:08:20.193761+0000","last_undegraded":"2024-03-18T23:08:20.193761+0000","last_fullsized":"2024-03-18T23:08:20.193761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:58.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914258+0000","last_change":"2024-03-18T23:07:14.062563+0000","last_active":"2024-03-18T23:08:20.914258+0000","last_peered":"2024-03-18T23:08:20.914258+0000","last_clean":"2024-03-18T23:08:20.914258+0000","last_became_active":"2024-03-18T23:07:14.062232+0000","last_became_peered":"2024-03-18T23:07:14.062232+0000","last_unstale":"2024-03-18T23:08:20.914258+0000","last_undegraded":"2024-03-18T23:08:20.914258+0000","last_fullsized":"2024-03-18T23:08:20.914258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:06.055127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.190995+0000","last_change":"2024-03-18T23:07:13.505103+0000","last_active":"2024-03-18T23:08:20.190995+0000","last_peered":"2024-03-18T23:08:20.190995+0000","last_clean":"2024-03-18T23:08:20.190995+0000","last_became_active":"2024-03-18T23:07:13.504764+0000","last_became_peered":"2024-03-18T23:07:13.504764+0000","last_unstale":"2024-03-18T23:08:20.190995+0000","last_undegraded":"2024-03-18T23:08:20.190995+0000","last_fullsized":"2024-03-18T23:08:20.190995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:07:44.998932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.192977+0000","last_change":"2024-03-18T23:07:14.066440+0000","last_active":"2024-03-18T23:08:20.192977+0000","last_peered":"2024-03-18T23:08:20.192977+0000","last_clean":"2024-03-18T23:08:20.192977+0000","last_became_active":"2024-03-18T23:07:14.066100+0000","last_became_peered":"2024-03-18T23:07:14.066100+0000","last_unstale":"2024-03-18T23:08:20.192977+0000","last_undegraded":"2024-03-18T23:08:20.192977+0000","last_fullsized":"2024-03-18T23:08:20.192977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:56:27.662447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:23.817097+0000","last_change":"2024-03-18T23:07:13.501952+0000","last_active":"2024-03-18T23:08:23.817097+0000","last_peered":"2024-03-18T23:08:23.817097+0000","last_clean":"2024-03-18T23:08:23.817097+0000","last_became_active":"2024-03-18T23:07:13.501628+0000","last_became_peered":"2024-03-18T23:07:13.501628+0000","last_unstale":"2024-03-18T23:08:23.817097+0000","last_undegraded":"2024-03-18T23:08:23.817097+0000","last_fullsized":"2024-03-18T23:08:23.817097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:09.046133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191391+0000","last_change":"2024-03-18T23:07:14.075917+0000","last_active":"2024-03-18T23:08:20.191391+0000","last_peered":"2024-03-18T23:08:20.191391+0000","last_clean":"2024-03-18T23:08:20.191391+0000","last_became_active":"2024-03-18T23:07:14.075522+0000","last_became_peered":"2024-03-18T23:07:14.075522+0000","last_unstale":"2024-03-18T23:08:20.191391+0000","last_undegraded":"2024-03-18T23:08:20.191391+0000","last_fullsized":"2024-03-18T23:08:20.191391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:28.108526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156388+0000","last_change":"2024-03-18T23:07:13.507271+0000","last_active":"2024-03-18T23:08:21.156388+0000","last_peered":"2024-03-18T23:08:21.156388+0000","last_clean":"2024-03-18T23:08:21.156388+0000","last_became_active":"2024-03-18T23:07:13.506771+0000","last_became_peered":"2024-03-18T23:07:13.506771+0000","last_unstale":"2024-03-18T23:08:21.156388+0000","last_undegraded":"2024-03-18T23:08:21.156388+0000","last_fullsized":"2024-03-18T23:08:21.156388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:23:08.552922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197093+0000","last_change":"2024-03-18T23:07:14.066838+0000","last_active":"2024-03-18T23:08:20.197093+0000","last_peered":"2024-03-18T23:08:20.197093+0000","last_clean":"2024-03-18T23:08:20.197093+0000","last_became_active":"2024-03-18T23:07:14.066550+0000","last_became_peered":"2024-03-18T23:07:14.066550+0000","last_unstale":"2024-03-18T23:08:20.197093+0000","last_undegraded":"2024-03-18T23:08:20.197093+0000","last_fullsized":"2024-03-18T23:08:20.197093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:59:19.048074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191578+0000","last_change":"2024-03-18T23:07:13.511861+0000","last_active":"2024-03-18T23:08:20.191578+0000","last_peered":"2024-03-18T23:08:20.191578+0000","last_clean":"2024-03-18T23:08:20.191578+0000","last_became_active":"2024-03-18T23:07:13.511458+0000","last_became_peered":"2024-03-18T23:07:13.511458+0000","last_unstale":"2024-03-18T23:08:20.191578+0000","last_undegraded":"2024-03-18T23:08:20.191578+0000","last_fullsized":"2024-03-18T23:08:20.191578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:03.301572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197862+0000","last_change":"2024-03-18T23:08:20.197862+0000","last_active":"2024-03-18T23:08:20.197862+0000","last_peered":"2024-03-18T23:08:20.197862+0000","last_clean":"2024-03-18T23:08:20.197862+0000","last_became_active":"2024-03-18T23:08:20.197586+0000","last_became_peered":"2024-03-18T23:08:20.197586+0000","last_unstale":"2024-03-18T23:08:20.197862+0000","last_undegraded":"2024-03-18T23:08:20.197862+0000","last_fullsized":"2024-03-18T23:08:20.197862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:53:48.730143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":186,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:24.818182+0000","last_change":"2024-03-18T23:07:13.505399+0000","last_active":"2024-03-18T23:08:24.818182+0000","last_peered":"2024-03-18T23:08:24.818182+0000","last_clean":"2024-03-18T23:08:24.818182+0000","last_became_active":"2024-03-18T23:07:13.505034+0000","last_became_peered":"2024-03-18T23:07:13.505034+0000","last_unstale":"2024-03-18T23:08:24.818182+0000","last_undegraded":"2024-03-18T23:08:24.818182+0000","last_fullsized":"2024-03-18T23:08:24.818182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:52.665189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":141,"num_read_kb":127,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197258+0000","last_change":"2024-03-18T23:07:14.059215+0000","last_active":"2024-03-18T23:08:20.197258+0000","last_peered":"2024-03-18T23:08:20.197258+0000","last_clean":"2024-03-18T23:08:20.197258+0000","last_became_active":"2024-03-18T23:07:14.058904+0000","last_became_peered":"2024-03-18T23:07:14.058904+0000","last_unstale":"2024-03-18T23:08:20.197258+0000","last_undegraded":"2024-03-18T23:08:20.197258+0000","last_fullsized":"2024-03-18T23:08:20.197258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:28:38.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198460+0000","last_change":"2024-03-18T23:07:13.503163+0000","last_active":"2024-03-18T23:08:20.198460+0000","last_peered":"2024-03-18T23:08:20.198460+0000","last_clean":"2024-03-18T23:08:20.198460+0000","last_became_active":"2024-03-18T23:07:13.502816+0000","last_became_peered":"2024-03-18T23:07:13.502816+0000","last_unstale":"2024-03-18T23:08:20.198460+0000","last_undegraded":"2024-03-18T23:08:20.198460+0000","last_fullsized":"2024-03-18T23:08:20.198460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:38:23.950178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193676+0000","last_change":"2024-03-18T23:07:14.057680+0000","last_active":"2024-03-18T23:08:20.193676+0000","last_peered":"2024-03-18T23:08:20.193676+0000","last_clean":"2024-03-18T23:08:20.193676+0000","last_became_active":"2024-03-18T23:07:14.057344+0000","last_became_peered":"2024-03-18T23:07:14.057344+0000","last_unstale":"2024-03-18T23:08:20.193676+0000","last_undegraded":"2024-03-18T23:08:20.193676+0000","last_fullsized":"2024-03-18T23:08:20.193676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:40.616782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191249+0000","last_change":"2024-03-18T23:07:13.500994+0000","last_active":"2024-03-18T23:08:20.191249+0000","last_peered":"2024-03-18T23:08:20.191249+0000","last_clean":"2024-03-18T23:08:20.191249+0000","last_became_active":"2024-03-18T23:07:13.500678+0000","last_became_peered":"2024-03-18T23:07:13.500678+0000","last_unstale":"2024-03-18T23:08:20.191249+0000","last_undegraded":"2024-03-18T23:08:20.191249+0000","last_fullsized":"2024-03-18T23:08:20.191249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:41:30.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156328+0000","last_change":"2024-03-18T23:07:14.073945+0000","last_active":"2024-03-18T23:08:21.156328+0000","last_peered":"2024-03-18T23:08:21.156328+0000","last_clean":"2024-03-18T23:08:21.156328+0000","last_became_active":"2024-03-18T23:07:14.073561+0000","last_became_peered":"2024-03-18T23:07:14.073561+0000","last_unstale":"2024-03-18T23:08:21.156328+0000","last_undegraded":"2024-03-18T23:08:21.156328+0000","last_fullsized":"2024-03-18T23:08:21.156328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:33:26.514969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.834210+0000","last_change":"2024-03-18T23:07:13.502121+0000","last_active":"2024-03-18T23:08:20.834210+0000","last_peered":"2024-03-18T23:08:20.834210+0000","last_clean":"2024-03-18T23:08:20.834210+0000","last_became_active":"2024-03-18T23:07:13.501796+0000","last_became_peered":"2024-03-18T23:07:13.501796+0000","last_unstale":"2024-03-18T23:08:20.834210+0000","last_undegraded":"2024-03-18T23:08:20.834210+0000","last_fullsized":"2024-03-18T23:08:20.834210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:02.348416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156060+0000","last_change":"2024-03-18T23:07:14.074062+0000","last_active":"2024-03-18T23:08:21.156060+0000","last_peered":"2024-03-18T23:08:21.156060+0000","last_clean":"2024-03-18T23:08:21.156060+0000","last_became_active":"2024-03-18T23:07:14.073657+0000","last_became_peered":"2024-03-18T23:07:14.073657+0000","last_unstale":"2024-03-18T23:08:21.156060+0000","last_undegraded":"2024-03-18T23:08:21.156060+0000","last_fullsized":"2024-03-18T23:08:21.156060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:28.103535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196821+0000","last_change":"2024-03-18T23:07:13.509334+0000","last_active":"2024-03-18T23:08:20.196821+0000","last_peered":"2024-03-18T23:08:20.196821+0000","last_clean":"2024-03-18T23:08:20.196821+0000","last_became_active":"2024-03-18T23:07:13.509036+0000","last_became_peered":"2024-03-18T23:07:13.509036+0000","last_unstale":"2024-03-18T23:08:20.196821+0000","last_undegraded":"2024-03-18T23:08:20.196821+0000","last_fullsized":"2024-03-18T23:08:20.196821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:14:27.935984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.199330+0000","last_change":"2024-03-18T23:08:20.199330+0000","last_active":"2024-03-18T23:08:20.199330+0000","last_peered":"2024-03-18T23:08:20.199330+0000","last_clean":"2024-03-18T23:08:20.199330+0000","last_became_active":"2024-03-18T23:08:20.198944+0000","last_became_peered":"2024-03-18T23:08:20.198944+0000","last_unstale":"2024-03-18T23:08:20.199330+0000","last_undegraded":"2024-03-18T23:08:20.199330+0000","last_fullsized":"2024-03-18T23:08:20.199330+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:02.125728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197512+0000","last_change":"2024-03-18T23:07:13.499616+0000","last_active":"2024-03-18T23:08:20.197512+0000","last_peered":"2024-03-18T23:08:20.197512+0000","last_clean":"2024-03-18T23:08:20.197512+0000","last_became_active":"2024-03-18T23:07:13.499129+0000","last_became_peered":"2024-03-18T23:07:13.499129+0000","last_unstale":"2024-03-18T23:08:20.197512+0000","last_undegraded":"2024-03-18T23:08:20.197512+0000","last_fullsized":"2024-03-18T23:08:20.197512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:28.699813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195142+0000","last_change":"2024-03-18T23:07:14.066305+0000","last_active":"2024-03-18T23:08:20.195142+0000","last_peered":"2024-03-18T23:08:20.195142+0000","last_clean":"2024-03-18T23:08:20.195142+0000","last_became_active":"2024-03-18T23:07:14.065952+0000","last_became_peered":"2024-03-18T23:07:14.065952+0000","last_unstale":"2024-03-18T23:08:20.195142+0000","last_undegraded":"2024-03-18T23:08:20.195142+0000","last_fullsized":"2024-03-18T23:08:20.195142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:06:00.718689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.199809+0000","last_change":"2024-03-18T23:08:20.199809+0000","last_active":"2024-03-18T23:08:20.199809+0000","last_peered":"2024-03-18T23:08:20.199809+0000","last_clean":"2024-03-18T23:08:20.199809+0000","last_became_active":"2024-03-18T23:08:20.199388+0000","last_became_peered":"2024-03-18T23:08:20.199388+0000","last_unstale":"2024-03-18T23:08:20.199809+0000","last_undegraded":"2024-03-18T23:08:20.199809+0000","last_fullsized":"2024-03-18T23:08:20.199809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:57.016118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813765+0000","last_change":"2024-03-18T23:07:14.072979+0000","last_active":"2024-03-18T23:08:20.813765+0000","last_peered":"2024-03-18T23:08:20.813765+0000","last_clean":"2024-03-18T23:08:20.813765+0000","last_became_active":"2024-03-18T23:07:14.072663+0000","last_became_peered":"2024-03-18T23:07:14.072663+0000","last_unstale":"2024-03-18T23:08:20.813765+0000","last_undegraded":"2024-03-18T23:08:20.813765+0000","last_fullsized":"2024-03-18T23:08:20.813765+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:19.916681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156174+0000","last_change":"2024-03-18T23:07:13.506789+0000","last_active":"2024-03-18T23:08:21.156174+0000","last_peered":"2024-03-18T23:08:21.156174+0000","last_clean":"2024-03-18T23:08:21.156174+0000","last_became_active":"2024-03-18T23:07:13.506412+0000","last_became_peered":"2024-03-18T23:07:13.506412+0000","last_unstale":"2024-03-18T23:08:21.156174+0000","last_undegraded":"2024-03-18T23:08:21.156174+0000","last_fullsized":"2024-03-18T23:08:21.156174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:37.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813835+0000","last_change":"2024-03-18T23:07:14.071419+0000","last_active":"2024-03-18T23:08:20.813835+0000","last_peered":"2024-03-18T23:08:20.813835+0000","last_clean":"2024-03-18T23:08:20.813835+0000","last_became_active":"2024-03-18T23:07:14.071071+0000","last_became_peered":"2024-03-18T23:07:14.071071+0000","last_unstale":"2024-03-18T23:08:20.813835+0000","last_undegraded":"2024-03-18T23:08:20.813835+0000","last_fullsized":"2024-03-18T23:08:20.813835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:25.657312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191129+0000","last_change":"2024-03-18T23:07:14.065418+0000","last_active":"2024-03-18T23:08:20.191129+0000","last_peered":"2024-03-18T23:08:20.191129+0000","last_clean":"2024-03-18T23:08:20.191129+0000","last_became_active":"2024-03-18T23:07:14.065057+0000","last_became_peered":"2024-03-18T23:07:14.065057+0000","last_unstale":"2024-03-18T23:08:20.191129+0000","last_undegraded":"2024-03-18T23:08:20.191129+0000","last_fullsized":"2024-03-18T23:08:20.191129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:23:29.531861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:24.838361+0000","last_change":"2024-03-18T23:07:13.510913+0000","last_active":"2024-03-18T23:08:24.838361+0000","last_peered":"2024-03-18T23:08:24.838361+0000","last_clean":"2024-03-18T23:08:24.838361+0000","last_became_active":"2024-03-18T23:07:13.510563+0000","last_became_peered":"2024-03-18T23:07:13.510563+0000","last_unstale":"2024-03-18T23:08:24.838361+0000","last_undegraded":"2024-03-18T23:08:24.838361+0000","last_fullsized":"2024-03-18T23:08:24.838361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:15.798492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914471+0000","last_change":"2024-03-18T23:07:14.065203+0000","last_active":"2024-03-18T23:08:20.914471+0000","last_peered":"2024-03-18T23:08:20.914471+0000","last_clean":"2024-03-18T23:08:20.914471+0000","last_became_active":"2024-03-18T23:07:14.064924+0000","last_became_peered":"2024-03-18T23:07:14.064924+0000","last_unstale":"2024-03-18T23:08:20.914471+0000","last_undegraded":"2024-03-18T23:08:20.914471+0000","last_fullsized":"2024-03-18T23:08:20.914471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:32:11.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196526+0000","last_change":"2024-03-18T23:07:13.495244+0000","last_active":"2024-03-18T23:08:20.196526+0000","last_peered":"2024-03-18T23:08:20.196526+0000","last_clean":"2024-03-18T23:08:20.196526+0000","last_became_active":"2024-03-18T23:07:13.494879+0000","last_became_peered":"2024-03-18T23:07:13.494879+0000","last_unstale":"2024-03-18T23:08:20.196526+0000","last_undegraded":"2024-03-18T23:08:20.196526+0000","last_fullsized":"2024-03-18T23:08:20.196526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:14:51.684956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156148+0000","last_change":"2024-03-18T23:06:03.931253+0000","last_active":"2024-03-18T23:08:21.156148+0000","last_peered":"2024-03-18T23:08:21.156148+0000","last_clean":"2024-03-18T23:08:21.156148+0000","last_became_active":"2024-03-18T23:06:03.930687+0000","last_became_peered":"2024-03-18T23:06:03.930687+0000","last_unstale":"2024-03-18T23:08:21.156148+0000","last_undegraded":"2024-03-18T23:08:21.156148+0000","last_fullsized":"2024-03-18T23:08:21.156148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_clean_scrub_stamp":"2024-03-18T23:02:45.345904+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:08:47.199331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193558+0000","last_change":"2024-03-18T23:07:14.065457+0000","last_active":"2024-03-18T23:08:20.193558+0000","last_peered":"2024-03-18T23:08:20.193558+0000","last_clean":"2024-03-18T23:08:20.193558+0000","last_became_active":"2024-03-18T23:07:14.065117+0000","last_became_peered":"2024-03-18T23:07:14.065117+0000","last_unstale":"2024-03-18T23:08:20.193558+0000","last_undegraded":"2024-03-18T23:08:20.193558+0000","last_fullsized":"2024-03-18T23:08:20.193558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:43.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156254+0000","last_change":"2024-03-18T23:07:13.501074+0000","last_active":"2024-03-18T23:08:21.156254+0000","last_peered":"2024-03-18T23:08:21.156254+0000","last_clean":"2024-03-18T23:08:21.156254+0000","last_became_active":"2024-03-18T23:07:13.500549+0000","last_became_peered":"2024-03-18T23:07:13.500549+0000","last_unstale":"2024-03-18T23:08:21.156254+0000","last_undegraded":"2024-03-18T23:08:21.156254+0000","last_fullsized":"2024-03-18T23:08:21.156254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:53:04.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.203253+0000","last_change":"2024-03-18T23:08:20.202849+0000","last_active":"2024-03-18T23:08:20.203253+0000","last_peered":"2024-03-18T23:08:20.203253+0000","last_clean":"2024-03-18T23:08:20.203253+0000","last_became_active":"2024-03-18T23:08:20.200049+0000","last_became_peered":"2024-03-18T23:08:20.200049+0000","last_unstale":"2024-03-18T23:08:20.203253+0000","last_undegraded":"2024-03-18T23:08:20.203253+0000","last_fullsized":"2024-03-18T23:08:20.203253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:15.255158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194280+0000","last_change":"2024-03-18T23:07:14.058971+0000","last_active":"2024-03-18T23:08:20.194280+0000","last_peered":"2024-03-18T23:08:20.194280+0000","last_clean":"2024-03-18T23:08:20.194280+0000","last_became_active":"2024-03-18T23:07:14.058619+0000","last_became_peered":"2024-03-18T23:07:14.058619+0000","last_unstale":"2024-03-18T23:08:20.194280+0000","last_undegraded":"2024-03-18T23:08:20.194280+0000","last_fullsized":"2024-03-18T23:08:20.194280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:55:37.814180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193312+0000","last_change":"2024-03-18T23:07:13.500594+0000","last_active":"2024-03-18T23:08:20.193312+0000","last_peered":"2024-03-18T23:08:20.193312+0000","last_clean":"2024-03-18T23:08:20.193312+0000","last_became_active":"2024-03-18T23:07:13.499223+0000","last_became_peered":"2024-03-18T23:07:13.499223+0000","last_unstale":"2024-03-18T23:08:20.193312+0000","last_undegraded":"2024-03-18T23:08:20.193312+0000","last_fullsized":"2024-03-18T23:08:20.193312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:27.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914748+0000","last_change":"2024-03-18T23:07:14.062569+0000","last_active":"2024-03-18T23:08:20.914748+0000","last_peered":"2024-03-18T23:08:20.914748+0000","last_clean":"2024-03-18T23:08:20.914748+0000","last_became_active":"2024-03-18T23:07:14.062231+0000","last_became_peered":"2024-03-18T23:07:14.062231+0000","last_unstale":"2024-03-18T23:08:20.914748+0000","last_undegraded":"2024-03-18T23:08:20.914748+0000","last_fullsized":"2024-03-18T23:08:20.914748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:03.987133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196751+0000","last_change":"2024-03-18T23:07:13.502513+0000","last_active":"2024-03-18T23:08:20.196751+0000","last_peered":"2024-03-18T23:08:20.196751+0000","last_clean":"2024-03-18T23:08:20.196751+0000","last_became_active":"2024-03-18T23:07:13.502005+0000","last_became_peered":"2024-03-18T23:07:13.502005+0000","last_unstale":"2024-03-18T23:08:20.196751+0000","last_undegraded":"2024-03-18T23:08:20.196751+0000","last_fullsized":"2024-03-18T23:08:20.196751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:05:10.395264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196946+0000","last_change":"2024-03-18T23:07:14.067346+0000","last_active":"2024-03-18T23:08:20.196946+0000","last_peered":"2024-03-18T23:08:20.196946+0000","last_clean":"2024-03-18T23:08:20.196946+0000","last_became_active":"2024-03-18T23:07:14.066390+0000","last_became_peered":"2024-03-18T23:07:14.066390+0000","last_unstale":"2024-03-18T23:08:20.196946+0000","last_undegraded":"2024-03-18T23:08:20.196946+0000","last_fullsized":"2024-03-18T23:08:20.196946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:07.679527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191095+0000","last_change":"2024-03-18T23:07:13.509887+0000","last_active":"2024-03-18T23:08:20.191095+0000","last_peered":"2024-03-18T23:08:20.191095+0000","last_clean":"2024-03-18T23:08:20.191095+0000","last_became_active":"2024-03-18T23:07:13.509631+0000","last_became_peered":"2024-03-18T23:07:13.509631+0000","last_unstale":"2024-03-18T23:08:20.191095+0000","last_undegraded":"2024-03-18T23:08:20.191095+0000","last_fullsized":"2024-03-18T23:08:20.191095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:55.782893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197153+0000","last_change":"2024-03-18T23:07:14.070297+0000","last_active":"2024-03-18T23:08:20.197153+0000","last_peered":"2024-03-18T23:08:20.197153+0000","last_clean":"2024-03-18T23:08:20.197153+0000","last_became_active":"2024-03-18T23:07:14.069696+0000","last_became_peered":"2024-03-18T23:07:14.069696+0000","last_unstale":"2024-03-18T23:08:20.197153+0000","last_undegraded":"2024-03-18T23:08:20.197153+0000","last_fullsized":"2024-03-18T23:08:20.197153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:47.448499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914339+0000","last_change":"2024-03-18T23:07:13.497436+0000","last_active":"2024-03-18T23:08:20.914339+0000","last_peered":"2024-03-18T23:08:20.914339+0000","last_clean":"2024-03-18T23:08:20.914339+0000","last_became_active":"2024-03-18T23:07:13.497090+0000","last_became_peered":"2024-03-18T23:07:13.497090+0000","last_unstale":"2024-03-18T23:08:20.914339+0000","last_undegraded":"2024-03-18T23:08:20.914339+0000","last_fullsized":"2024-03-18T23:08:20.914339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:33.496595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194905+0000","last_change":"2024-03-18T23:07:14.061399+0000","last_active":"2024-03-18T23:08:20.194905+0000","last_peered":"2024-03-18T23:08:20.194905+0000","last_clean":"2024-03-18T23:08:20.194905+0000","last_became_active":"2024-03-18T23:07:14.060753+0000","last_became_peered":"2024-03-18T23:07:14.060753+0000","last_unstale":"2024-03-18T23:08:20.194905+0000","last_undegraded":"2024-03-18T23:08:20.194905+0000","last_fullsized":"2024-03-18T23:08:20.194905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:56:21.688951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198294+0000","last_change":"2024-03-18T23:07:13.506678+0000","last_active":"2024-03-18T23:08:20.198294+0000","last_peered":"2024-03-18T23:08:20.198294+0000","last_clean":"2024-03-18T23:08:20.198294+0000","last_became_active":"2024-03-18T23:07:13.506374+0000","last_became_peered":"2024-03-18T23:07:13.506374+0000","last_unstale":"2024-03-18T23:08:20.198294+0000","last_undegraded":"2024-03-18T23:08:20.198294+0000","last_fullsized":"2024-03-18T23:08:20.198294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:03.265106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914480+0000","last_change":"2024-03-18T23:07:14.070134+0000","last_active":"2024-03-18T23:08:20.914480+0000","last_peered":"2024-03-18T23:08:20.914480+0000","last_clean":"2024-03-18T23:08:20.914480+0000","last_became_active":"2024-03-18T23:07:14.069785+0000","last_became_peered":"2024-03-18T23:07:14.069785+0000","last_unstale":"2024-03-18T23:08:20.914480+0000","last_undegraded":"2024-03-18T23:08:20.914480+0000","last_fullsized":"2024-03-18T23:08:20.914480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:17:34.975546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197107+0000","last_change":"2024-03-18T23:07:13.502516+0000","last_active":"2024-03-18T23:08:20.197107+0000","last_peered":"2024-03-18T23:08:20.197107+0000","last_clean":"2024-03-18T23:08:20.197107+0000","last_became_active":"2024-03-18T23:07:13.502141+0000","last_became_peered":"2024-03-18T23:07:13.502141+0000","last_unstale":"2024-03-18T23:08:20.197107+0000","last_undegraded":"2024-03-18T23:08:20.197107+0000","last_fullsized":"2024-03-18T23:08:20.197107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:10.328673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156289+0000","last_change":"2024-03-18T23:07:14.070465+0000","last_active":"2024-03-18T23:08:21.156289+0000","last_peered":"2024-03-18T23:08:21.156289+0000","last_clean":"2024-03-18T23:08:21.156289+0000","last_became_active":"2024-03-18T23:07:14.070158+0000","last_became_peered":"2024-03-18T23:07:14.070158+0000","last_unstale":"2024-03-18T23:08:21.156289+0000","last_undegraded":"2024-03-18T23:08:21.156289+0000","last_fullsized":"2024-03-18T23:08:21.156289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:53.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.207516+0000","last_change":"2024-03-18T23:08:20.207100+0000","last_active":"2024-03-18T23:08:20.207516+0000","last_peered":"2024-03-18T23:08:20.207516+0000","last_clean":"2024-03-18T23:08:20.207516+0000","last_became_active":"2024-03-18T23:08:20.200265+0000","last_became_peered":"2024-03-18T23:08:20.200265+0000","last_unstale":"2024-03-18T23:08:20.207516+0000","last_undegraded":"2024-03-18T23:08:20.207516+0000","last_fullsized":"2024-03-18T23:08:20.207516+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:26.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156539+0000","last_change":"2024-03-18T23:07:14.071489+0000","last_active":"2024-03-18T23:08:21.156539+0000","last_peered":"2024-03-18T23:08:21.156539+0000","last_clean":"2024-03-18T23:08:21.156539+0000","last_became_active":"2024-03-18T23:07:14.071175+0000","last_became_peered":"2024-03-18T23:07:14.071175+0000","last_unstale":"2024-03-18T23:08:21.156539+0000","last_undegraded":"2024-03-18T23:08:21.156539+0000","last_fullsized":"2024-03-18T23:08:21.156539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:39.813451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191331+0000","last_change":"2024-03-18T23:07:13.500233+0000","last_active":"2024-03-18T23:08:20.191331+0000","last_peered":"2024-03-18T23:08:20.191331+0000","last_clean":"2024-03-18T23:08:20.191331+0000","last_became_active":"2024-03-18T23:07:13.499870+0000","last_became_peered":"2024-03-18T23:07:13.499870+0000","last_unstale":"2024-03-18T23:08:20.191331+0000","last_undegraded":"2024-03-18T23:08:20.191331+0000","last_fullsized":"2024-03-18T23:08:20.191331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:45.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156366+0000","last_change":"2024-03-18T23:07:14.065225+0000","last_active":"2024-03-18T23:08:21.156366+0000","last_peered":"2024-03-18T23:08:21.156366+0000","last_clean":"2024-03-18T23:08:21.156366+0000","last_became_active":"2024-03-18T23:07:14.064881+0000","last_became_peered":"2024-03-18T23:07:14.064881+0000","last_unstale":"2024-03-18T23:08:21.156366+0000","last_undegraded":"2024-03-18T23:08:21.156366+0000","last_fullsized":"2024-03-18T23:08:21.156366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:31:27.921072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194054+0000","last_change":"2024-03-18T23:07:13.502565+0000","last_active":"2024-03-18T23:08:20.194054+0000","last_peered":"2024-03-18T23:08:20.194054+0000","last_clean":"2024-03-18T23:08:20.194054+0000","last_became_active":"2024-03-18T23:07:13.501777+0000","last_became_peered":"2024-03-18T23:07:13.501777+0000","last_unstale":"2024-03-18T23:08:20.194054+0000","last_undegraded":"2024-03-18T23:08:20.194054+0000","last_fullsized":"2024-03-18T23:08:20.194054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:04.482546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156448+0000","last_change":"2024-03-18T23:07:14.068691+0000","last_active":"2024-03-18T23:08:21.156448+0000","last_peered":"2024-03-18T23:08:21.156448+0000","last_clean":"2024-03-18T23:08:21.156448+0000","last_became_active":"2024-03-18T23:07:14.068268+0000","last_became_peered":"2024-03-18T23:07:14.068268+0000","last_unstale":"2024-03-18T23:08:21.156448+0000","last_undegraded":"2024-03-18T23:08:21.156448+0000","last_fullsized":"2024-03-18T23:08:21.156448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:55:56.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194938+0000","last_change":"2024-03-18T23:07:14.061079+0000","last_active":"2024-03-18T23:08:20.194938+0000","last_peered":"2024-03-18T23:08:20.194938+0000","last_clean":"2024-03-18T23:08:20.194938+0000","last_became_active":"2024-03-18T23:07:14.060379+0000","last_became_peered":"2024-03-18T23:07:14.060379+0000","last_unstale":"2024-03-18T23:08:20.194938+0000","last_undegraded":"2024-03-18T23:08:20.194938+0000","last_fullsized":"2024-03-18T23:08:20.194938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:17.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198584+0000","last_change":"2024-03-18T23:07:14.069349+0000","last_active":"2024-03-18T23:08:20.198584+0000","last_peered":"2024-03-18T23:08:20.198584+0000","last_clean":"2024-03-18T23:08:20.198584+0000","last_became_active":"2024-03-18T23:07:14.068777+0000","last_became_peered":"2024-03-18T23:07:14.068777+0000","last_unstale":"2024-03-18T23:08:20.198584+0000","last_undegraded":"2024-03-18T23:08:20.198584+0000","last_fullsized":"2024-03-18T23:08:20.198584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:32.509613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198632+0000","last_change":"2024-03-18T23:07:14.062981+0000","last_active":"2024-03-18T23:08:20.198632+0000","last_peered":"2024-03-18T23:08:20.198632+0000","last_clean":"2024-03-18T23:08:20.198632+0000","last_became_active":"2024-03-18T23:07:14.061986+0000","last_became_peered":"2024-03-18T23:07:14.061986+0000","last_unstale":"2024-03-18T23:08:20.198632+0000","last_undegraded":"2024-03-18T23:08:20.198632+0000","last_fullsized":"2024-03-18T23:08:20.198632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:46.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196767+0000","last_change":"2024-03-18T23:07:14.069200+0000","last_active":"2024-03-18T23:08:20.196767+0000","last_peered":"2024-03-18T23:08:20.196767+0000","last_clean":"2024-03-18T23:08:20.196767+0000","last_became_active":"2024-03-18T23:07:14.068893+0000","last_became_peered":"2024-03-18T23:07:14.068893+0000","last_unstale":"2024-03-18T23:08:20.196767+0000","last_undegraded":"2024-03-18T23:08:20.196767+0000","last_fullsized":"2024-03-18T23:08:20.196767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:34.488251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194309+0000","last_change":"2024-03-18T23:07:14.065429+0000","last_active":"2024-03-18T23:08:20.194309+0000","last_peered":"2024-03-18T23:08:20.194309+0000","last_clean":"2024-03-18T23:08:20.194309+0000","last_became_active":"2024-03-18T23:07:14.065083+0000","last_became_peered":"2024-03-18T23:07:14.065083+0000","last_unstale":"2024-03-18T23:08:20.194309+0000","last_undegraded":"2024-03-18T23:08:20.194309+0000","last_fullsized":"2024-03-18T23:08:20.194309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:37.317538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195420+0000","last_change":"2024-03-18T23:07:14.059155+0000","last_active":"2024-03-18T23:08:20.195420+0000","last_peered":"2024-03-18T23:08:20.195420+0000","last_clean":"2024-03-18T23:08:20.195420+0000","last_became_active":"2024-03-18T23:07:14.058800+0000","last_became_peered":"2024-03-18T23:07:14.058800+0000","last_unstale":"2024-03-18T23:08:20.195420+0000","last_undegraded":"2024-03-18T23:08:20.195420+0000","last_fullsized":"2024-03-18T23:08:20.195420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:32.994668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198894+0000","last_change":"2024-03-18T23:07:14.065351+0000","last_active":"2024-03-18T23:08:20.198894+0000","last_peered":"2024-03-18T23:08:20.198894+0000","last_clean":"2024-03-18T23:08:20.198894+0000","last_became_active":"2024-03-18T23:07:14.065012+0000","last_became_peered":"2024-03-18T23:07:14.065012+0000","last_unstale":"2024-03-18T23:08:20.198894+0000","last_undegraded":"2024-03-18T23:08:20.198894+0000","last_fullsized":"2024-03-18T23:08:20.198894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:35.452474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914703+0000","last_change":"2024-03-18T23:07:14.067179+0000","last_active":"2024-03-18T23:08:20.914703+0000","last_peered":"2024-03-18T23:08:20.914703+0000","last_clean":"2024-03-18T23:08:20.914703+0000","last_became_active":"2024-03-18T23:07:14.066899+0000","last_became_peered":"2024-03-18T23:07:14.066899+0000","last_unstale":"2024-03-18T23:08:20.914703+0000","last_undegraded":"2024-03-18T23:08:20.914703+0000","last_fullsized":"2024-03-18T23:08:20.914703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:02.180512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191105+0000","last_change":"2024-03-18T23:07:14.061089+0000","last_active":"2024-03-18T23:08:20.191105+0000","last_peered":"2024-03-18T23:08:20.191105+0000","last_clean":"2024-03-18T23:08:20.191105+0000","last_became_active":"2024-03-18T23:07:14.060787+0000","last_became_peered":"2024-03-18T23:07:14.060787+0000","last_unstale":"2024-03-18T23:08:20.191105+0000","last_undegraded":"2024-03-18T23:08:20.191105+0000","last_fullsized":"2024-03-18T23:08:20.191105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:47:52.029524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914349+0000","last_change":"2024-03-18T23:07:14.058888+0000","last_active":"2024-03-18T23:08:20.914349+0000","last_peered":"2024-03-18T23:08:20.914349+0000","last_clean":"2024-03-18T23:08:20.914349+0000","last_became_active":"2024-03-18T23:07:14.058576+0000","last_became_peered":"2024-03-18T23:07:14.058576+0000","last_unstale":"2024-03-18T23:08:20.914349+0000","last_undegraded":"2024-03-18T23:08:20.914349+0000","last_fullsized":"2024-03-18T23:08:20.914349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:27.121268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191307+0000","last_change":"2024-03-18T23:07:14.076698+0000","last_active":"2024-03-18T23:08:20.191307+0000","last_peered":"2024-03-18T23:08:20.191307+0000","last_clean":"2024-03-18T23:08:20.191307+0000","last_became_active":"2024-03-18T23:07:14.076156+0000","last_became_peered":"2024-03-18T23:07:14.076156+0000","last_unstale":"2024-03-18T23:08:20.191307+0000","last_undegraded":"2024-03-18T23:08:20.191307+0000","last_fullsized":"2024-03-18T23:08:20.191307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:56:01.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193795+0000","last_change":"2024-03-18T23:07:14.057774+0000","last_active":"2024-03-18T23:08:20.193795+0000","last_peered":"2024-03-18T23:08:20.193795+0000","last_clean":"2024-03-18T23:08:20.193795+0000","last_became_active":"2024-03-18T23:07:14.057456+0000","last_became_peered":"2024-03-18T23:07:14.057456+0000","last_unstale":"2024-03-18T23:08:20.193795+0000","last_undegraded":"2024-03-18T23:08:20.193795+0000","last_fullsized":"2024-03-18T23:08:20.193795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:33.742347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195066+0000","last_change":"2024-03-18T23:07:14.066689+0000","last_active":"2024-03-18T23:08:20.195066+0000","last_peered":"2024-03-18T23:08:20.195066+0000","last_clean":"2024-03-18T23:08:20.195066+0000","last_became_active":"2024-03-18T23:07:14.066358+0000","last_became_peered":"2024-03-18T23:07:14.066358+0000","last_unstale":"2024-03-18T23:08:20.195066+0000","last_undegraded":"2024-03-18T23:08:20.195066+0000","last_fullsized":"2024-03-18T23:08:20.195066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:29.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63304,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":422,"num_read_kb":280,"num_write":155,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285597,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":468891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1119653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":469105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972394,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234032,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1061346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":468494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":527199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":469765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":468805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1597440,"data_stored":1060639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:08:29.158 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T23:08:29.414 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:29 smithi196 ceph-mon[67089]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:29.756 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:29 smithi132 ceph-mon[32020]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 61 B/s, 0 objects/s recovering 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:08:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:29 smithi144 ceph-mon[66781]: from='mgr.14152 172.21.15.132:0/2542042627' entity='mgr.x' 2024-03-18T23:08:30.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 -- 172.21.15.132:0/3545146486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994106530 msgr2=0x7f399410e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 --2- 172.21.15.132:0/3545146486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994106530 0x7f399410e100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3984009990 tx=0x7f3984009ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 -- 172.21.15.132:0/3545146486 shutdown_connections 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 --2- 172.21.15.132:0/3545146486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39941014a0 0x7f3994101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 --2- 172.21.15.132:0/3545146486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994106530 0x7f399410e100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 --2- 172.21.15.132:0/3545146486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3994101e50 0x7f3994105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 -- 172.21.15.132:0/3545146486 >> 172.21.15.132:0/3545146486 conn(0x7f3994077230 msgr2=0x7f3994077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 -- 172.21.15.132:0/3545146486 shutdown_connections 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.962+0000 7f3998a40700 1 -- 172.21.15.132:0/3545146486 wait complete. 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 Processor -- start 2024-03-18T23:08:30.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 -- start start 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39941014a0 0x7f3994197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3994106530 0x7f3994198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3994111510 con 0x7f3994101e50 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3994111390 con 0x7f3994106530 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3998a40700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3994111690 con 0x7f39941014a0 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3991d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3991d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51654/0 (socket says 172.21.15.132:51654) 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.963+0000 7f3991d9b700 1 -- 172.21.15.132:0/582846836 learned_addr learned my addr 172.21.15.132:0/582846836 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f399259c700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39941014a0 0x7f3994197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3992d9d700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3994106530 0x7f3994198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 -- 172.21.15.132:0/582846836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39941014a0 msgr2=0x7f3994197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39941014a0 0x7f3994197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 -- 172.21.15.132:0/582846836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3994106530 msgr2=0x7f3994198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3994106530 0x7f3994198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 -- 172.21.15.132:0/582846836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941a6f10 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3991d9b700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988004a30 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f398800cca0 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941a7200 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880056c0 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.964+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941a76f0 con 0x7f3994101e50 2024-03-18T23:08:30.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.965+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3994102930 con 0x7f3994101e50 2024-03-18T23:08:30.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.966+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f398800c7c0 con 0x7f3994101e50 2024-03-18T23:08:30.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.967+0000 7f39837fe700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 0x7f397c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:30.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.967+0000 7f399259c700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 0x7f397c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:30.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.967+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f39880cef50 con 0x7f3994101e50 2024-03-18T23:08:30.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.967+0000 7f399259c700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 0x7f397c0a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f398c00f540 tx=0x7f398c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:30.971 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:30.971+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3988053740 con 0x7f3994101e50 2024-03-18T23:08:31.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.164+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3994062750 con 0x7f397c0a2920 2024-03-18T23:08:31.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.171+0000 7f39837fe700 1 -- 172.21.15.132:0/582846836 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258706 (secure 0 0 0) 0x7f3994062750 con 0x7f397c0a2920 2024-03-18T23:08:31.176 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:31.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.175+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 msgr2=0x7f397c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:31.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.175+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 0x7f397c0a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f398c00f540 tx=0x7f398c00b040 comp rx=0 tx=0).stop 2024-03-18T23:08:31.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.175+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 msgr2=0x7f3994197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:31.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.175+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f398800e9f0 tx=0x7f398800ed00 comp rx=0 tx=0).stop 2024-03-18T23:08:31.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 shutdown_connections 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f397c0a2920 0x7f397c0a4de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3994106530 0x7f3994198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39941014a0 0x7f3994197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 --2- 172.21.15.132:0/582846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3994101e50 0x7f3994197c40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 >> 172.21.15.132:0/582846836 conn(0x7f3994077230 msgr2=0x7f39940ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 shutdown_connections 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:31.176+0000 7f3998a40700 1 -- 172.21.15.132:0/582846836 wait complete. 2024-03-18T23:08:31.178 INFO:teuthology.orchestra.run.smithi132.stderr:dumped all 2024-03-18T23:08:31.784 INFO:teuthology.orchestra.run.smithi132.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-03-18T23:08:31.062994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653906,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":583,"num_read_kb":409,"num_write":338,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181340,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371092,"kb_used_data":13548,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570732,"statfs":{"total":1151940427776,"available":1151560429568,"internally_reserved":0,"allocated":13873152,"data_stored":7581162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.871056"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195321+0000","last_change":"2024-03-18T23:07:14.066593+0000","last_active":"2024-03-18T23:08:20.195321+0000","last_peered":"2024-03-18T23:08:20.195321+0000","last_clean":"2024-03-18T23:08:20.195321+0000","last_became_active":"2024-03-18T23:07:14.066064+0000","last_became_peered":"2024-03-18T23:07:14.066064+0000","last_unstale":"2024-03-18T23:08:20.195321+0000","last_undegraded":"2024-03-18T23:08:20.195321+0000","last_fullsized":"2024-03-18T23:08:20.195321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:31:48.122520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196884+0000","last_change":"2024-03-18T23:07:14.070484+0000","last_active":"2024-03-18T23:08:20.196884+0000","last_peered":"2024-03-18T23:08:20.196884+0000","last_clean":"2024-03-18T23:08:20.196884+0000","last_became_active":"2024-03-18T23:07:14.070129+0000","last_became_peered":"2024-03-18T23:07:14.070129+0000","last_unstale":"2024-03-18T23:08:20.196884+0000","last_undegraded":"2024-03-18T23:08:20.196884+0000","last_fullsized":"2024-03-18T23:08:20.196884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:13:44.453172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196727+0000","last_change":"2024-03-18T23:07:14.064923+0000","last_active":"2024-03-18T23:08:20.196727+0000","last_peered":"2024-03-18T23:08:20.196727+0000","last_clean":"2024-03-18T23:08:20.196727+0000","last_became_active":"2024-03-18T23:07:14.064513+0000","last_became_peered":"2024-03-18T23:07:14.064513+0000","last_unstale":"2024-03-18T23:08:20.196727+0000","last_undegraded":"2024-03-18T23:08:20.196727+0000","last_fullsized":"2024-03-18T23:08:20.196727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:33.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.200142+0000","last_change":"2024-03-18T23:08:20.199613+0000","last_active":"2024-03-18T23:08:20.200142+0000","last_peered":"2024-03-18T23:08:20.200142+0000","last_clean":"2024-03-18T23:08:20.200142+0000","last_became_active":"2024-03-18T23:08:20.199160+0000","last_became_peered":"2024-03-18T23:08:20.199160+0000","last_unstale":"2024-03-18T23:08:20.200142+0000","last_undegraded":"2024-03-18T23:08:20.200142+0000","last_fullsized":"2024-03-18T23:08:20.200142+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:42:20.345400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197015+0000","last_change":"2024-03-18T23:07:14.058801+0000","last_active":"2024-03-18T23:08:20.197015+0000","last_peered":"2024-03-18T23:08:20.197015+0000","last_clean":"2024-03-18T23:08:20.197015+0000","last_became_active":"2024-03-18T23:07:14.058481+0000","last_became_peered":"2024-03-18T23:07:14.058481+0000","last_unstale":"2024-03-18T23:08:20.197015+0000","last_undegraded":"2024-03-18T23:08:20.197015+0000","last_fullsized":"2024-03-18T23:08:20.197015+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:55.420890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813500+0000","last_change":"2024-03-18T23:07:14.067320+0000","last_active":"2024-03-18T23:08:20.813500+0000","last_peered":"2024-03-18T23:08:20.813500+0000","last_clean":"2024-03-18T23:08:20.813500+0000","last_became_active":"2024-03-18T23:07:14.066987+0000","last_became_peered":"2024-03-18T23:07:14.066987+0000","last_unstale":"2024-03-18T23:08:20.813500+0000","last_undegraded":"2024-03-18T23:08:20.813500+0000","last_fullsized":"2024-03-18T23:08:20.813500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:52:33.006026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156112+0000","last_change":"2024-03-18T23:07:14.065320+0000","last_active":"2024-03-18T23:08:21.156112+0000","last_peered":"2024-03-18T23:08:21.156112+0000","last_clean":"2024-03-18T23:08:21.156112+0000","last_became_active":"2024-03-18T23:07:14.064280+0000","last_became_peered":"2024-03-18T23:07:14.064280+0000","last_unstale":"2024-03-18T23:08:21.156112+0000","last_undegraded":"2024-03-18T23:08:21.156112+0000","last_fullsized":"2024-03-18T23:08:21.156112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:07.838038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.815904+0000","last_change":"2024-03-18T23:08:20.815124+0000","last_active":"2024-03-18T23:08:20.815904+0000","last_peered":"2024-03-18T23:08:20.815904+0000","last_clean":"2024-03-18T23:08:20.815904+0000","last_became_active":"2024-03-18T23:08:20.814510+0000","last_became_peered":"2024-03-18T23:08:20.814510+0000","last_unstale":"2024-03-18T23:08:20.815904+0000","last_undegraded":"2024-03-18T23:08:20.815904+0000","last_fullsized":"2024-03-18T23:08:20.815904+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:02:18.592085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191604+0000","last_change":"2024-03-18T23:07:14.072829+0000","last_active":"2024-03-18T23:08:20.191604+0000","last_peered":"2024-03-18T23:08:20.191604+0000","last_clean":"2024-03-18T23:08:20.191604+0000","last_became_active":"2024-03-18T23:07:14.072493+0000","last_became_peered":"2024-03-18T23:07:14.072493+0000","last_unstale":"2024-03-18T23:08:20.191604+0000","last_undegraded":"2024-03-18T23:08:20.191604+0000","last_fullsized":"2024-03-18T23:08:20.191604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:58:12.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813792+0000","last_change":"2024-03-18T23:07:14.067119+0000","last_active":"2024-03-18T23:08:20.813792+0000","last_peered":"2024-03-18T23:08:20.813792+0000","last_clean":"2024-03-18T23:08:20.813792+0000","last_became_active":"2024-03-18T23:07:14.066738+0000","last_became_peered":"2024-03-18T23:07:14.066738+0000","last_unstale":"2024-03-18T23:08:20.813792+0000","last_undegraded":"2024-03-18T23:08:20.813792+0000","last_fullsized":"2024-03-18T23:08:20.813792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:55.011692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156513+0000","last_change":"2024-03-18T23:07:14.071791+0000","last_active":"2024-03-18T23:08:21.156513+0000","last_peered":"2024-03-18T23:08:21.156513+0000","last_clean":"2024-03-18T23:08:21.156513+0000","last_became_active":"2024-03-18T23:07:14.068031+0000","last_became_peered":"2024-03-18T23:07:14.068031+0000","last_unstale":"2024-03-18T23:08:21.156513+0000","last_undegraded":"2024-03-18T23:08:21.156513+0000","last_fullsized":"2024-03-18T23:08:21.156513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:42:56.142512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196919+0000","last_change":"2024-03-18T23:07:14.071712+0000","last_active":"2024-03-18T23:08:20.196919+0000","last_peered":"2024-03-18T23:08:20.196919+0000","last_clean":"2024-03-18T23:08:20.196919+0000","last_became_active":"2024-03-18T23:07:14.069595+0000","last_became_peered":"2024-03-18T23:07:14.069595+0000","last_unstale":"2024-03-18T23:08:20.196919+0000","last_undegraded":"2024-03-18T23:08:20.196919+0000","last_fullsized":"2024-03-18T23:08:20.196919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:10:21.521681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.815159+0000","last_change":"2024-03-18T23:08:20.815159+0000","last_active":"2024-03-18T23:08:20.815159+0000","last_peered":"2024-03-18T23:08:20.815159+0000","last_clean":"2024-03-18T23:08:20.815159+0000","last_became_active":"2024-03-18T23:08:20.814675+0000","last_became_peered":"2024-03-18T23:08:20.814675+0000","last_unstale":"2024-03-18T23:08:20.815159+0000","last_undegraded":"2024-03-18T23:08:20.815159+0000","last_fullsized":"2024-03-18T23:08:20.815159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:08:36.453847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196696+0000","last_change":"2024-03-18T23:07:14.070755+0000","last_active":"2024-03-18T23:08:20.196696+0000","last_peered":"2024-03-18T23:08:20.196696+0000","last_clean":"2024-03-18T23:08:20.196696+0000","last_became_active":"2024-03-18T23:07:14.070126+0000","last_became_peered":"2024-03-18T23:07:14.070126+0000","last_unstale":"2024-03-18T23:08:20.196696+0000","last_undegraded":"2024-03-18T23:08:20.196696+0000","last_fullsized":"2024-03-18T23:08:20.196696+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:05.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196553+0000","last_change":"2024-03-18T23:07:14.070166+0000","last_active":"2024-03-18T23:08:20.196553+0000","last_peered":"2024-03-18T23:08:20.196553+0000","last_clean":"2024-03-18T23:08:20.196553+0000","last_became_active":"2024-03-18T23:07:14.069693+0000","last_became_peered":"2024-03-18T23:07:14.069693+0000","last_unstale":"2024-03-18T23:08:20.196553+0000","last_undegraded":"2024-03-18T23:08:20.196553+0000","last_fullsized":"2024-03-18T23:08:20.196553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:03:55.616894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191091+0000","last_change":"2024-03-18T23:07:14.076732+0000","last_active":"2024-03-18T23:08:20.191091+0000","last_peered":"2024-03-18T23:08:20.191091+0000","last_clean":"2024-03-18T23:08:20.191091+0000","last_became_active":"2024-03-18T23:07:14.076143+0000","last_became_peered":"2024-03-18T23:07:14.076143+0000","last_unstale":"2024-03-18T23:08:20.191091+0000","last_undegraded":"2024-03-18T23:08:20.191091+0000","last_fullsized":"2024-03-18T23:08:20.191091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:46.888524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195486+0000","last_change":"2024-03-18T23:07:14.066388+0000","last_active":"2024-03-18T23:08:20.195486+0000","last_peered":"2024-03-18T23:08:20.195486+0000","last_clean":"2024-03-18T23:08:20.195486+0000","last_became_active":"2024-03-18T23:07:14.065626+0000","last_became_peered":"2024-03-18T23:07:14.065626+0000","last_unstale":"2024-03-18T23:08:20.195486+0000","last_undegraded":"2024-03-18T23:08:20.195486+0000","last_fullsized":"2024-03-18T23:08:20.195486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:08:18.200855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193357+0000","last_change":"2024-03-18T23:07:14.068730+0000","last_active":"2024-03-18T23:08:20.193357+0000","last_peered":"2024-03-18T23:08:20.193357+0000","last_clean":"2024-03-18T23:08:20.193357+0000","last_became_active":"2024-03-18T23:07:14.068420+0000","last_became_peered":"2024-03-18T23:07:14.068420+0000","last_unstale":"2024-03-18T23:08:20.193357+0000","last_undegraded":"2024-03-18T23:08:20.193357+0000","last_fullsized":"2024-03-18T23:08:20.193357+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:41:34.588823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198509+0000","last_change":"2024-03-18T23:07:14.061415+0000","last_active":"2024-03-18T23:08:20.198509+0000","last_peered":"2024-03-18T23:08:20.198509+0000","last_clean":"2024-03-18T23:08:20.198509+0000","last_became_active":"2024-03-18T23:07:14.061102+0000","last_became_peered":"2024-03-18T23:07:14.061102+0000","last_unstale":"2024-03-18T23:08:20.198509+0000","last_undegraded":"2024-03-18T23:08:20.198509+0000","last_fullsized":"2024-03-18T23:08:20.198509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:06:08.056213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.192985+0000","last_change":"2024-03-18T23:07:13.506404+0000","last_active":"2024-03-18T23:08:20.192985+0000","last_peered":"2024-03-18T23:08:20.192985+0000","last_clean":"2024-03-18T23:08:20.192985+0000","last_became_active":"2024-03-18T23:07:13.506091+0000","last_became_peered":"2024-03-18T23:07:13.506091+0000","last_unstale":"2024-03-18T23:08:20.192985+0000","last_undegraded":"2024-03-18T23:08:20.192985+0000","last_fullsized":"2024-03-18T23:08:20.192985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:34:26.916813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914771+0000","last_change":"2024-03-18T23:07:14.061990+0000","last_active":"2024-03-18T23:08:20.914771+0000","last_peered":"2024-03-18T23:08:20.914771+0000","last_clean":"2024-03-18T23:08:20.914771+0000","last_became_active":"2024-03-18T23:07:14.061581+0000","last_became_peered":"2024-03-18T23:07:14.061581+0000","last_unstale":"2024-03-18T23:08:20.914771+0000","last_undegraded":"2024-03-18T23:08:20.914771+0000","last_fullsized":"2024-03-18T23:08:20.914771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:52:37.726342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914806+0000","last_change":"2024-03-18T23:07:13.509727+0000","last_active":"2024-03-18T23:08:20.914806+0000","last_peered":"2024-03-18T23:08:20.914806+0000","last_clean":"2024-03-18T23:08:20.914806+0000","last_became_active":"2024-03-18T23:07:13.509296+0000","last_became_peered":"2024-03-18T23:07:13.509296+0000","last_unstale":"2024-03-18T23:08:20.914806+0000","last_undegraded":"2024-03-18T23:08:20.914806+0000","last_fullsized":"2024-03-18T23:08:20.914806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:15.060387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197181+0000","last_change":"2024-03-18T23:07:14.066038+0000","last_active":"2024-03-18T23:08:20.197181+0000","last_peered":"2024-03-18T23:08:20.197181+0000","last_clean":"2024-03-18T23:08:20.197181+0000","last_became_active":"2024-03-18T23:07:14.065449+0000","last_became_peered":"2024-03-18T23:07:14.065449+0000","last_unstale":"2024-03-18T23:08:20.197181+0000","last_undegraded":"2024-03-18T23:08:20.197181+0000","last_fullsized":"2024-03-18T23:08:20.197181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:53:07.619275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197350+0000","last_change":"2024-03-18T23:07:13.503422+0000","last_active":"2024-03-18T23:08:20.197350+0000","last_peered":"2024-03-18T23:08:20.197350+0000","last_clean":"2024-03-18T23:08:20.197350+0000","last_became_active":"2024-03-18T23:07:13.503068+0000","last_became_peered":"2024-03-18T23:07:13.503068+0000","last_unstale":"2024-03-18T23:08:20.197350+0000","last_undegraded":"2024-03-18T23:08:20.197350+0000","last_fullsized":"2024-03-18T23:08:20.197350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:35.369317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191189+0000","last_change":"2024-03-18T23:07:14.073092+0000","last_active":"2024-03-18T23:08:20.191189+0000","last_peered":"2024-03-18T23:08:20.191189+0000","last_clean":"2024-03-18T23:08:20.191189+0000","last_became_active":"2024-03-18T23:07:14.070303+0000","last_became_peered":"2024-03-18T23:07:14.070303+0000","last_unstale":"2024-03-18T23:08:20.191189+0000","last_undegraded":"2024-03-18T23:08:20.191189+0000","last_fullsized":"2024-03-18T23:08:20.191189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:09.992019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197336+0000","last_change":"2024-03-18T23:07:13.505420+0000","last_active":"2024-03-18T23:08:20.197336+0000","last_peered":"2024-03-18T23:08:20.197336+0000","last_clean":"2024-03-18T23:08:20.197336+0000","last_became_active":"2024-03-18T23:07:13.504995+0000","last_became_peered":"2024-03-18T23:07:13.504995+0000","last_unstale":"2024-03-18T23:08:20.197336+0000","last_undegraded":"2024-03-18T23:08:20.197336+0000","last_fullsized":"2024-03-18T23:08:20.197336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:40:50.988318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191481+0000","last_change":"2024-03-18T23:07:14.071254+0000","last_active":"2024-03-18T23:08:20.191481+0000","last_peered":"2024-03-18T23:08:20.191481+0000","last_clean":"2024-03-18T23:08:20.191481+0000","last_became_active":"2024-03-18T23:07:14.070767+0000","last_became_peered":"2024-03-18T23:07:14.070767+0000","last_unstale":"2024-03-18T23:08:20.191481+0000","last_undegraded":"2024-03-18T23:08:20.191481+0000","last_fullsized":"2024-03-18T23:08:20.191481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:05:36.639093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:25.819718+0000","last_change":"2024-03-18T23:07:13.505020+0000","last_active":"2024-03-18T23:08:25.819718+0000","last_peered":"2024-03-18T23:08:25.819718+0000","last_clean":"2024-03-18T23:08:25.819718+0000","last_became_active":"2024-03-18T23:07:13.504355+0000","last_became_peered":"2024-03-18T23:07:13.504355+0000","last_unstale":"2024-03-18T23:08:25.819718+0000","last_undegraded":"2024-03-18T23:08:25.819718+0000","last_fullsized":"2024-03-18T23:08:25.819718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:03.184513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196979+0000","last_change":"2024-03-18T23:07:14.063156+0000","last_active":"2024-03-18T23:08:20.196979+0000","last_peered":"2024-03-18T23:08:20.196979+0000","last_clean":"2024-03-18T23:08:20.196979+0000","last_became_active":"2024-03-18T23:07:14.062801+0000","last_became_peered":"2024-03-18T23:07:14.062801+0000","last_unstale":"2024-03-18T23:08:20.196979+0000","last_undegraded":"2024-03-18T23:08:20.196979+0000","last_fullsized":"2024-03-18T23:08:20.196979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:49:27.426387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813469+0000","last_change":"2024-03-18T23:07:13.501538+0000","last_active":"2024-03-18T23:08:20.813469+0000","last_peered":"2024-03-18T23:08:20.813469+0000","last_clean":"2024-03-18T23:08:20.813469+0000","last_became_active":"2024-03-18T23:07:13.501160+0000","last_became_peered":"2024-03-18T23:07:13.501160+0000","last_unstale":"2024-03-18T23:08:20.813469+0000","last_undegraded":"2024-03-18T23:08:20.813469+0000","last_fullsized":"2024-03-18T23:08:20.813469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:11.319636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191446+0000","last_change":"2024-03-18T23:07:14.069345+0000","last_active":"2024-03-18T23:08:20.191446+0000","last_peered":"2024-03-18T23:08:20.191446+0000","last_clean":"2024-03-18T23:08:20.191446+0000","last_became_active":"2024-03-18T23:07:14.068320+0000","last_became_peered":"2024-03-18T23:07:14.068320+0000","last_unstale":"2024-03-18T23:08:20.191446+0000","last_undegraded":"2024-03-18T23:08:20.191446+0000","last_fullsized":"2024-03-18T23:08:20.191446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:36.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193761+0000","last_change":"2024-03-18T23:07:13.507538+0000","last_active":"2024-03-18T23:08:20.193761+0000","last_peered":"2024-03-18T23:08:20.193761+0000","last_clean":"2024-03-18T23:08:20.193761+0000","last_became_active":"2024-03-18T23:07:13.506563+0000","last_became_peered":"2024-03-18T23:07:13.506563+0000","last_unstale":"2024-03-18T23:08:20.193761+0000","last_undegraded":"2024-03-18T23:08:20.193761+0000","last_fullsized":"2024-03-18T23:08:20.193761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:22:58.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914258+0000","last_change":"2024-03-18T23:07:14.062563+0000","last_active":"2024-03-18T23:08:20.914258+0000","last_peered":"2024-03-18T23:08:20.914258+0000","last_clean":"2024-03-18T23:08:20.914258+0000","last_became_active":"2024-03-18T23:07:14.062232+0000","last_became_peered":"2024-03-18T23:07:14.062232+0000","last_unstale":"2024-03-18T23:08:20.914258+0000","last_undegraded":"2024-03-18T23:08:20.914258+0000","last_fullsized":"2024-03-18T23:08:20.914258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:43:06.055127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.190995+0000","last_change":"2024-03-18T23:07:13.505103+0000","last_active":"2024-03-18T23:08:20.190995+0000","last_peered":"2024-03-18T23:08:20.190995+0000","last_clean":"2024-03-18T23:08:20.190995+0000","last_became_active":"2024-03-18T23:07:13.504764+0000","last_became_peered":"2024-03-18T23:07:13.504764+0000","last_unstale":"2024-03-18T23:08:20.190995+0000","last_undegraded":"2024-03-18T23:08:20.190995+0000","last_fullsized":"2024-03-18T23:08:20.190995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:07:44.998932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.192977+0000","last_change":"2024-03-18T23:07:14.066440+0000","last_active":"2024-03-18T23:08:20.192977+0000","last_peered":"2024-03-18T23:08:20.192977+0000","last_clean":"2024-03-18T23:08:20.192977+0000","last_became_active":"2024-03-18T23:07:14.066100+0000","last_became_peered":"2024-03-18T23:07:14.066100+0000","last_unstale":"2024-03-18T23:08:20.192977+0000","last_undegraded":"2024-03-18T23:08:20.192977+0000","last_fullsized":"2024-03-18T23:08:20.192977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:56:27.662447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:28.822264+0000","last_change":"2024-03-18T23:07:13.501952+0000","last_active":"2024-03-18T23:08:28.822264+0000","last_peered":"2024-03-18T23:08:28.822264+0000","last_clean":"2024-03-18T23:08:28.822264+0000","last_became_active":"2024-03-18T23:07:13.501628+0000","last_became_peered":"2024-03-18T23:07:13.501628+0000","last_unstale":"2024-03-18T23:08:28.822264+0000","last_undegraded":"2024-03-18T23:08:28.822264+0000","last_fullsized":"2024-03-18T23:08:28.822264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:09.046133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191391+0000","last_change":"2024-03-18T23:07:14.075917+0000","last_active":"2024-03-18T23:08:20.191391+0000","last_peered":"2024-03-18T23:08:20.191391+0000","last_clean":"2024-03-18T23:08:20.191391+0000","last_became_active":"2024-03-18T23:07:14.075522+0000","last_became_peered":"2024-03-18T23:07:14.075522+0000","last_unstale":"2024-03-18T23:08:20.191391+0000","last_undegraded":"2024-03-18T23:08:20.191391+0000","last_fullsized":"2024-03-18T23:08:20.191391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:28.108526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156388+0000","last_change":"2024-03-18T23:07:13.507271+0000","last_active":"2024-03-18T23:08:21.156388+0000","last_peered":"2024-03-18T23:08:21.156388+0000","last_clean":"2024-03-18T23:08:21.156388+0000","last_became_active":"2024-03-18T23:07:13.506771+0000","last_became_peered":"2024-03-18T23:07:13.506771+0000","last_unstale":"2024-03-18T23:08:21.156388+0000","last_undegraded":"2024-03-18T23:08:21.156388+0000","last_fullsized":"2024-03-18T23:08:21.156388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:23:08.552922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197093+0000","last_change":"2024-03-18T23:07:14.066838+0000","last_active":"2024-03-18T23:08:20.197093+0000","last_peered":"2024-03-18T23:08:20.197093+0000","last_clean":"2024-03-18T23:08:20.197093+0000","last_became_active":"2024-03-18T23:07:14.066550+0000","last_became_peered":"2024-03-18T23:07:14.066550+0000","last_unstale":"2024-03-18T23:08:20.197093+0000","last_undegraded":"2024-03-18T23:08:20.197093+0000","last_fullsized":"2024-03-18T23:08:20.197093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:59:19.048074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191578+0000","last_change":"2024-03-18T23:07:13.511861+0000","last_active":"2024-03-18T23:08:20.191578+0000","last_peered":"2024-03-18T23:08:20.191578+0000","last_clean":"2024-03-18T23:08:20.191578+0000","last_became_active":"2024-03-18T23:07:13.511458+0000","last_became_peered":"2024-03-18T23:07:13.511458+0000","last_unstale":"2024-03-18T23:08:20.191578+0000","last_undegraded":"2024-03-18T23:08:20.191578+0000","last_fullsized":"2024-03-18T23:08:20.191578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:03.301572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197862+0000","last_change":"2024-03-18T23:08:20.197862+0000","last_active":"2024-03-18T23:08:20.197862+0000","last_peered":"2024-03-18T23:08:20.197862+0000","last_clean":"2024-03-18T23:08:20.197862+0000","last_became_active":"2024-03-18T23:08:20.197586+0000","last_became_peered":"2024-03-18T23:08:20.197586+0000","last_unstale":"2024-03-18T23:08:20.197862+0000","last_undegraded":"2024-03-18T23:08:20.197862+0000","last_fullsized":"2024-03-18T23:08:20.197862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:53:48.730143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":196,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:29.823286+0000","last_change":"2024-03-18T23:07:13.505399+0000","last_active":"2024-03-18T23:08:29.823286+0000","last_peered":"2024-03-18T23:08:29.823286+0000","last_clean":"2024-03-18T23:08:29.823286+0000","last_became_active":"2024-03-18T23:07:13.505034+0000","last_became_peered":"2024-03-18T23:07:13.505034+0000","last_unstale":"2024-03-18T23:08:29.823286+0000","last_undegraded":"2024-03-18T23:08:29.823286+0000","last_fullsized":"2024-03-18T23:08:29.823286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:52.665189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197258+0000","last_change":"2024-03-18T23:07:14.059215+0000","last_active":"2024-03-18T23:08:20.197258+0000","last_peered":"2024-03-18T23:08:20.197258+0000","last_clean":"2024-03-18T23:08:20.197258+0000","last_became_active":"2024-03-18T23:07:14.058904+0000","last_became_peered":"2024-03-18T23:07:14.058904+0000","last_unstale":"2024-03-18T23:08:20.197258+0000","last_undegraded":"2024-03-18T23:08:20.197258+0000","last_fullsized":"2024-03-18T23:08:20.197258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:28:38.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198460+0000","last_change":"2024-03-18T23:07:13.503163+0000","last_active":"2024-03-18T23:08:20.198460+0000","last_peered":"2024-03-18T23:08:20.198460+0000","last_clean":"2024-03-18T23:08:20.198460+0000","last_became_active":"2024-03-18T23:07:13.502816+0000","last_became_peered":"2024-03-18T23:07:13.502816+0000","last_unstale":"2024-03-18T23:08:20.198460+0000","last_undegraded":"2024-03-18T23:08:20.198460+0000","last_fullsized":"2024-03-18T23:08:20.198460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:38:23.950178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193676+0000","last_change":"2024-03-18T23:07:14.057680+0000","last_active":"2024-03-18T23:08:20.193676+0000","last_peered":"2024-03-18T23:08:20.193676+0000","last_clean":"2024-03-18T23:08:20.193676+0000","last_became_active":"2024-03-18T23:07:14.057344+0000","last_became_peered":"2024-03-18T23:07:14.057344+0000","last_unstale":"2024-03-18T23:08:20.193676+0000","last_undegraded":"2024-03-18T23:08:20.193676+0000","last_fullsized":"2024-03-18T23:08:20.193676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:21:40.616782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191249+0000","last_change":"2024-03-18T23:07:13.500994+0000","last_active":"2024-03-18T23:08:20.191249+0000","last_peered":"2024-03-18T23:08:20.191249+0000","last_clean":"2024-03-18T23:08:20.191249+0000","last_became_active":"2024-03-18T23:07:13.500678+0000","last_became_peered":"2024-03-18T23:07:13.500678+0000","last_unstale":"2024-03-18T23:08:20.191249+0000","last_undegraded":"2024-03-18T23:08:20.191249+0000","last_fullsized":"2024-03-18T23:08:20.191249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:41:30.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156328+0000","last_change":"2024-03-18T23:07:14.073945+0000","last_active":"2024-03-18T23:08:21.156328+0000","last_peered":"2024-03-18T23:08:21.156328+0000","last_clean":"2024-03-18T23:08:21.156328+0000","last_became_active":"2024-03-18T23:07:14.073561+0000","last_became_peered":"2024-03-18T23:07:14.073561+0000","last_unstale":"2024-03-18T23:08:21.156328+0000","last_undegraded":"2024-03-18T23:08:21.156328+0000","last_fullsized":"2024-03-18T23:08:21.156328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:33:26.514969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:25.839884+0000","last_change":"2024-03-18T23:07:13.502121+0000","last_active":"2024-03-18T23:08:25.839884+0000","last_peered":"2024-03-18T23:08:25.839884+0000","last_clean":"2024-03-18T23:08:25.839884+0000","last_became_active":"2024-03-18T23:07:13.501796+0000","last_became_peered":"2024-03-18T23:07:13.501796+0000","last_unstale":"2024-03-18T23:08:25.839884+0000","last_undegraded":"2024-03-18T23:08:25.839884+0000","last_fullsized":"2024-03-18T23:08:25.839884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:39:02.348416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156060+0000","last_change":"2024-03-18T23:07:14.074062+0000","last_active":"2024-03-18T23:08:21.156060+0000","last_peered":"2024-03-18T23:08:21.156060+0000","last_clean":"2024-03-18T23:08:21.156060+0000","last_became_active":"2024-03-18T23:07:14.073657+0000","last_became_peered":"2024-03-18T23:07:14.073657+0000","last_unstale":"2024-03-18T23:08:21.156060+0000","last_undegraded":"2024-03-18T23:08:21.156060+0000","last_fullsized":"2024-03-18T23:08:21.156060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:28.103535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196821+0000","last_change":"2024-03-18T23:07:13.509334+0000","last_active":"2024-03-18T23:08:20.196821+0000","last_peered":"2024-03-18T23:08:20.196821+0000","last_clean":"2024-03-18T23:08:20.196821+0000","last_became_active":"2024-03-18T23:07:13.509036+0000","last_became_peered":"2024-03-18T23:07:13.509036+0000","last_unstale":"2024-03-18T23:08:20.196821+0000","last_undegraded":"2024-03-18T23:08:20.196821+0000","last_fullsized":"2024-03-18T23:08:20.196821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:14:27.935984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.199330+0000","last_change":"2024-03-18T23:08:20.199330+0000","last_active":"2024-03-18T23:08:20.199330+0000","last_peered":"2024-03-18T23:08:20.199330+0000","last_clean":"2024-03-18T23:08:20.199330+0000","last_became_active":"2024-03-18T23:08:20.198944+0000","last_became_peered":"2024-03-18T23:08:20.198944+0000","last_unstale":"2024-03-18T23:08:20.199330+0000","last_undegraded":"2024-03-18T23:08:20.199330+0000","last_fullsized":"2024-03-18T23:08:20.199330+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:03:02.125728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197512+0000","last_change":"2024-03-18T23:07:13.499616+0000","last_active":"2024-03-18T23:08:20.197512+0000","last_peered":"2024-03-18T23:08:20.197512+0000","last_clean":"2024-03-18T23:08:20.197512+0000","last_became_active":"2024-03-18T23:07:13.499129+0000","last_became_peered":"2024-03-18T23:07:13.499129+0000","last_unstale":"2024-03-18T23:08:20.197512+0000","last_undegraded":"2024-03-18T23:08:20.197512+0000","last_fullsized":"2024-03-18T23:08:20.197512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:28.699813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195142+0000","last_change":"2024-03-18T23:07:14.066305+0000","last_active":"2024-03-18T23:08:20.195142+0000","last_peered":"2024-03-18T23:08:20.195142+0000","last_clean":"2024-03-18T23:08:20.195142+0000","last_became_active":"2024-03-18T23:07:14.065952+0000","last_became_peered":"2024-03-18T23:07:14.065952+0000","last_unstale":"2024-03-18T23:08:20.195142+0000","last_undegraded":"2024-03-18T23:08:20.195142+0000","last_fullsized":"2024-03-18T23:08:20.195142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:06:00.718689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.199809+0000","last_change":"2024-03-18T23:08:20.199809+0000","last_active":"2024-03-18T23:08:20.199809+0000","last_peered":"2024-03-18T23:08:20.199809+0000","last_clean":"2024-03-18T23:08:20.199809+0000","last_became_active":"2024-03-18T23:08:20.199388+0000","last_became_peered":"2024-03-18T23:08:20.199388+0000","last_unstale":"2024-03-18T23:08:20.199809+0000","last_undegraded":"2024-03-18T23:08:20.199809+0000","last_fullsized":"2024-03-18T23:08:20.199809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:33:57.016118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813765+0000","last_change":"2024-03-18T23:07:14.072979+0000","last_active":"2024-03-18T23:08:20.813765+0000","last_peered":"2024-03-18T23:08:20.813765+0000","last_clean":"2024-03-18T23:08:20.813765+0000","last_became_active":"2024-03-18T23:07:14.072663+0000","last_became_peered":"2024-03-18T23:07:14.072663+0000","last_unstale":"2024-03-18T23:08:20.813765+0000","last_undegraded":"2024-03-18T23:08:20.813765+0000","last_fullsized":"2024-03-18T23:08:20.813765+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:34:19.916681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156174+0000","last_change":"2024-03-18T23:07:13.506789+0000","last_active":"2024-03-18T23:08:21.156174+0000","last_peered":"2024-03-18T23:08:21.156174+0000","last_clean":"2024-03-18T23:08:21.156174+0000","last_became_active":"2024-03-18T23:07:13.506412+0000","last_became_peered":"2024-03-18T23:07:13.506412+0000","last_unstale":"2024-03-18T23:08:21.156174+0000","last_undegraded":"2024-03-18T23:08:21.156174+0000","last_fullsized":"2024-03-18T23:08:21.156174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:37.277886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.813835+0000","last_change":"2024-03-18T23:07:14.071419+0000","last_active":"2024-03-18T23:08:20.813835+0000","last_peered":"2024-03-18T23:08:20.813835+0000","last_clean":"2024-03-18T23:08:20.813835+0000","last_became_active":"2024-03-18T23:07:14.071071+0000","last_became_peered":"2024-03-18T23:07:14.071071+0000","last_unstale":"2024-03-18T23:08:20.813835+0000","last_undegraded":"2024-03-18T23:08:20.813835+0000","last_fullsized":"2024-03-18T23:08:20.813835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:25.657312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191129+0000","last_change":"2024-03-18T23:07:14.065418+0000","last_active":"2024-03-18T23:08:20.191129+0000","last_peered":"2024-03-18T23:08:20.191129+0000","last_clean":"2024-03-18T23:08:20.191129+0000","last_became_active":"2024-03-18T23:07:14.065057+0000","last_became_peered":"2024-03-18T23:07:14.065057+0000","last_unstale":"2024-03-18T23:08:20.191129+0000","last_undegraded":"2024-03-18T23:08:20.191129+0000","last_fullsized":"2024-03-18T23:08:20.191129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:23:29.531861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:24.838361+0000","last_change":"2024-03-18T23:07:13.510913+0000","last_active":"2024-03-18T23:08:24.838361+0000","last_peered":"2024-03-18T23:08:24.838361+0000","last_clean":"2024-03-18T23:08:24.838361+0000","last_became_active":"2024-03-18T23:07:13.510563+0000","last_became_peered":"2024-03-18T23:07:13.510563+0000","last_unstale":"2024-03-18T23:08:24.838361+0000","last_undegraded":"2024-03-18T23:08:24.838361+0000","last_fullsized":"2024-03-18T23:08:24.838361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:49:15.798492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914471+0000","last_change":"2024-03-18T23:07:14.065203+0000","last_active":"2024-03-18T23:08:20.914471+0000","last_peered":"2024-03-18T23:08:20.914471+0000","last_clean":"2024-03-18T23:08:20.914471+0000","last_became_active":"2024-03-18T23:07:14.064924+0000","last_became_peered":"2024-03-18T23:07:14.064924+0000","last_unstale":"2024-03-18T23:08:20.914471+0000","last_undegraded":"2024-03-18T23:08:20.914471+0000","last_fullsized":"2024-03-18T23:08:20.914471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:32:11.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196526+0000","last_change":"2024-03-18T23:07:13.495244+0000","last_active":"2024-03-18T23:08:20.196526+0000","last_peered":"2024-03-18T23:08:20.196526+0000","last_clean":"2024-03-18T23:08:20.196526+0000","last_became_active":"2024-03-18T23:07:13.494879+0000","last_became_peered":"2024-03-18T23:07:13.494879+0000","last_unstale":"2024-03-18T23:08:20.196526+0000","last_undegraded":"2024-03-18T23:08:20.196526+0000","last_fullsized":"2024-03-18T23:08:20.196526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:14:51.684956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156148+0000","last_change":"2024-03-18T23:06:03.931253+0000","last_active":"2024-03-18T23:08:21.156148+0000","last_peered":"2024-03-18T23:08:21.156148+0000","last_clean":"2024-03-18T23:08:21.156148+0000","last_became_active":"2024-03-18T23:06:03.930687+0000","last_became_peered":"2024-03-18T23:06:03.930687+0000","last_unstale":"2024-03-18T23:08:21.156148+0000","last_undegraded":"2024-03-18T23:08:21.156148+0000","last_fullsized":"2024-03-18T23:08:21.156148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:02:45.345904+0000","last_clean_scrub_stamp":"2024-03-18T23:02:45.345904+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:08:47.199331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193558+0000","last_change":"2024-03-18T23:07:14.065457+0000","last_active":"2024-03-18T23:08:20.193558+0000","last_peered":"2024-03-18T23:08:20.193558+0000","last_clean":"2024-03-18T23:08:20.193558+0000","last_became_active":"2024-03-18T23:07:14.065117+0000","last_became_peered":"2024-03-18T23:07:14.065117+0000","last_unstale":"2024-03-18T23:08:20.193558+0000","last_undegraded":"2024-03-18T23:08:20.193558+0000","last_fullsized":"2024-03-18T23:08:20.193558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:43.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156254+0000","last_change":"2024-03-18T23:07:13.501074+0000","last_active":"2024-03-18T23:08:21.156254+0000","last_peered":"2024-03-18T23:08:21.156254+0000","last_clean":"2024-03-18T23:08:21.156254+0000","last_became_active":"2024-03-18T23:07:13.500549+0000","last_became_peered":"2024-03-18T23:07:13.500549+0000","last_unstale":"2024-03-18T23:08:21.156254+0000","last_undegraded":"2024-03-18T23:08:21.156254+0000","last_fullsized":"2024-03-18T23:08:21.156254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:53:04.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.203253+0000","last_change":"2024-03-18T23:08:20.202849+0000","last_active":"2024-03-18T23:08:20.203253+0000","last_peered":"2024-03-18T23:08:20.203253+0000","last_clean":"2024-03-18T23:08:20.203253+0000","last_became_active":"2024-03-18T23:08:20.200049+0000","last_became_peered":"2024-03-18T23:08:20.200049+0000","last_unstale":"2024-03-18T23:08:20.203253+0000","last_undegraded":"2024-03-18T23:08:20.203253+0000","last_fullsized":"2024-03-18T23:08:20.203253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:26:15.255158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194280+0000","last_change":"2024-03-18T23:07:14.058971+0000","last_active":"2024-03-18T23:08:20.194280+0000","last_peered":"2024-03-18T23:08:20.194280+0000","last_clean":"2024-03-18T23:08:20.194280+0000","last_became_active":"2024-03-18T23:07:14.058619+0000","last_became_peered":"2024-03-18T23:07:14.058619+0000","last_unstale":"2024-03-18T23:08:20.194280+0000","last_undegraded":"2024-03-18T23:08:20.194280+0000","last_fullsized":"2024-03-18T23:08:20.194280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:55:37.814180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193312+0000","last_change":"2024-03-18T23:07:13.500594+0000","last_active":"2024-03-18T23:08:20.193312+0000","last_peered":"2024-03-18T23:08:20.193312+0000","last_clean":"2024-03-18T23:08:20.193312+0000","last_became_active":"2024-03-18T23:07:13.499223+0000","last_became_peered":"2024-03-18T23:07:13.499223+0000","last_unstale":"2024-03-18T23:08:20.193312+0000","last_undegraded":"2024-03-18T23:08:20.193312+0000","last_fullsized":"2024-03-18T23:08:20.193312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:05:27.269893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914748+0000","last_change":"2024-03-18T23:07:14.062569+0000","last_active":"2024-03-18T23:08:20.914748+0000","last_peered":"2024-03-18T23:08:20.914748+0000","last_clean":"2024-03-18T23:08:20.914748+0000","last_became_active":"2024-03-18T23:07:14.062231+0000","last_became_peered":"2024-03-18T23:07:14.062231+0000","last_unstale":"2024-03-18T23:08:20.914748+0000","last_undegraded":"2024-03-18T23:08:20.914748+0000","last_fullsized":"2024-03-18T23:08:20.914748+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:27:03.987133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196751+0000","last_change":"2024-03-18T23:07:13.502513+0000","last_active":"2024-03-18T23:08:20.196751+0000","last_peered":"2024-03-18T23:08:20.196751+0000","last_clean":"2024-03-18T23:08:20.196751+0000","last_became_active":"2024-03-18T23:07:13.502005+0000","last_became_peered":"2024-03-18T23:07:13.502005+0000","last_unstale":"2024-03-18T23:08:20.196751+0000","last_undegraded":"2024-03-18T23:08:20.196751+0000","last_fullsized":"2024-03-18T23:08:20.196751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:05:10.395264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196946+0000","last_change":"2024-03-18T23:07:14.067346+0000","last_active":"2024-03-18T23:08:20.196946+0000","last_peered":"2024-03-18T23:08:20.196946+0000","last_clean":"2024-03-18T23:08:20.196946+0000","last_became_active":"2024-03-18T23:07:14.066390+0000","last_became_peered":"2024-03-18T23:07:14.066390+0000","last_unstale":"2024-03-18T23:08:20.196946+0000","last_undegraded":"2024-03-18T23:08:20.196946+0000","last_fullsized":"2024-03-18T23:08:20.196946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:07.679527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191095+0000","last_change":"2024-03-18T23:07:13.509887+0000","last_active":"2024-03-18T23:08:20.191095+0000","last_peered":"2024-03-18T23:08:20.191095+0000","last_clean":"2024-03-18T23:08:20.191095+0000","last_became_active":"2024-03-18T23:07:13.509631+0000","last_became_peered":"2024-03-18T23:07:13.509631+0000","last_unstale":"2024-03-18T23:08:20.191095+0000","last_undegraded":"2024-03-18T23:08:20.191095+0000","last_fullsized":"2024-03-18T23:08:20.191095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:55.782893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197153+0000","last_change":"2024-03-18T23:07:14.070297+0000","last_active":"2024-03-18T23:08:20.197153+0000","last_peered":"2024-03-18T23:08:20.197153+0000","last_clean":"2024-03-18T23:08:20.197153+0000","last_became_active":"2024-03-18T23:07:14.069696+0000","last_became_peered":"2024-03-18T23:07:14.069696+0000","last_unstale":"2024-03-18T23:08:20.197153+0000","last_undegraded":"2024-03-18T23:08:20.197153+0000","last_fullsized":"2024-03-18T23:08:20.197153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:47.448499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914339+0000","last_change":"2024-03-18T23:07:13.497436+0000","last_active":"2024-03-18T23:08:20.914339+0000","last_peered":"2024-03-18T23:08:20.914339+0000","last_clean":"2024-03-18T23:08:20.914339+0000","last_became_active":"2024-03-18T23:07:13.497090+0000","last_became_peered":"2024-03-18T23:07:13.497090+0000","last_unstale":"2024-03-18T23:08:20.914339+0000","last_undegraded":"2024-03-18T23:08:20.914339+0000","last_fullsized":"2024-03-18T23:08:20.914339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:49:33.496595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194905+0000","last_change":"2024-03-18T23:07:14.061399+0000","last_active":"2024-03-18T23:08:20.194905+0000","last_peered":"2024-03-18T23:08:20.194905+0000","last_clean":"2024-03-18T23:08:20.194905+0000","last_became_active":"2024-03-18T23:07:14.060753+0000","last_became_peered":"2024-03-18T23:07:14.060753+0000","last_unstale":"2024-03-18T23:08:20.194905+0000","last_undegraded":"2024-03-18T23:08:20.194905+0000","last_fullsized":"2024-03-18T23:08:20.194905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:56:21.688951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198294+0000","last_change":"2024-03-18T23:07:13.506678+0000","last_active":"2024-03-18T23:08:20.198294+0000","last_peered":"2024-03-18T23:08:20.198294+0000","last_clean":"2024-03-18T23:08:20.198294+0000","last_became_active":"2024-03-18T23:07:13.506374+0000","last_became_peered":"2024-03-18T23:07:13.506374+0000","last_unstale":"2024-03-18T23:08:20.198294+0000","last_undegraded":"2024-03-18T23:08:20.198294+0000","last_fullsized":"2024-03-18T23:08:20.198294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:03.265106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914480+0000","last_change":"2024-03-18T23:07:14.070134+0000","last_active":"2024-03-18T23:08:20.914480+0000","last_peered":"2024-03-18T23:08:20.914480+0000","last_clean":"2024-03-18T23:08:20.914480+0000","last_became_active":"2024-03-18T23:07:14.069785+0000","last_became_peered":"2024-03-18T23:07:14.069785+0000","last_unstale":"2024-03-18T23:08:20.914480+0000","last_undegraded":"2024-03-18T23:08:20.914480+0000","last_fullsized":"2024-03-18T23:08:20.914480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:17:34.975546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.197107+0000","last_change":"2024-03-18T23:07:13.502516+0000","last_active":"2024-03-18T23:08:20.197107+0000","last_peered":"2024-03-18T23:08:20.197107+0000","last_clean":"2024-03-18T23:08:20.197107+0000","last_became_active":"2024-03-18T23:07:13.502141+0000","last_became_peered":"2024-03-18T23:07:13.502141+0000","last_unstale":"2024-03-18T23:08:20.197107+0000","last_undegraded":"2024-03-18T23:08:20.197107+0000","last_fullsized":"2024-03-18T23:08:20.197107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:10.328673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156289+0000","last_change":"2024-03-18T23:07:14.070465+0000","last_active":"2024-03-18T23:08:21.156289+0000","last_peered":"2024-03-18T23:08:21.156289+0000","last_clean":"2024-03-18T23:08:21.156289+0000","last_became_active":"2024-03-18T23:07:14.070158+0000","last_became_peered":"2024-03-18T23:07:14.070158+0000","last_unstale":"2024-03-18T23:08:21.156289+0000","last_undegraded":"2024-03-18T23:08:21.156289+0000","last_fullsized":"2024-03-18T23:08:21.156289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:53.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.207516+0000","last_change":"2024-03-18T23:08:20.207100+0000","last_active":"2024-03-18T23:08:20.207516+0000","last_peered":"2024-03-18T23:08:20.207516+0000","last_clean":"2024-03-18T23:08:20.207516+0000","last_became_active":"2024-03-18T23:08:20.200265+0000","last_became_peered":"2024-03-18T23:08:20.200265+0000","last_unstale":"2024-03-18T23:08:20.207516+0000","last_undegraded":"2024-03-18T23:08:20.207516+0000","last_fullsized":"2024-03-18T23:08:20.207516+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:26.516139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156539+0000","last_change":"2024-03-18T23:07:14.071489+0000","last_active":"2024-03-18T23:08:21.156539+0000","last_peered":"2024-03-18T23:08:21.156539+0000","last_clean":"2024-03-18T23:08:21.156539+0000","last_became_active":"2024-03-18T23:07:14.071175+0000","last_became_peered":"2024-03-18T23:07:14.071175+0000","last_unstale":"2024-03-18T23:08:21.156539+0000","last_undegraded":"2024-03-18T23:08:21.156539+0000","last_fullsized":"2024-03-18T23:08:21.156539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:39.813451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191331+0000","last_change":"2024-03-18T23:07:13.500233+0000","last_active":"2024-03-18T23:08:20.191331+0000","last_peered":"2024-03-18T23:08:20.191331+0000","last_clean":"2024-03-18T23:08:20.191331+0000","last_became_active":"2024-03-18T23:07:13.499870+0000","last_became_peered":"2024-03-18T23:07:13.499870+0000","last_unstale":"2024-03-18T23:08:20.191331+0000","last_undegraded":"2024-03-18T23:08:20.191331+0000","last_fullsized":"2024-03-18T23:08:20.191331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:40:45.851033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156366+0000","last_change":"2024-03-18T23:07:14.065225+0000","last_active":"2024-03-18T23:08:21.156366+0000","last_peered":"2024-03-18T23:08:21.156366+0000","last_clean":"2024-03-18T23:08:21.156366+0000","last_became_active":"2024-03-18T23:07:14.064881+0000","last_became_peered":"2024-03-18T23:07:14.064881+0000","last_unstale":"2024-03-18T23:08:21.156366+0000","last_undegraded":"2024-03-18T23:08:21.156366+0000","last_fullsized":"2024-03-18T23:08:21.156366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:31:27.921072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194054+0000","last_change":"2024-03-18T23:07:13.502565+0000","last_active":"2024-03-18T23:08:20.194054+0000","last_peered":"2024-03-18T23:08:20.194054+0000","last_clean":"2024-03-18T23:08:20.194054+0000","last_became_active":"2024-03-18T23:07:13.501777+0000","last_became_peered":"2024-03-18T23:07:13.501777+0000","last_unstale":"2024-03-18T23:08:20.194054+0000","last_undegraded":"2024-03-18T23:08:20.194054+0000","last_fullsized":"2024-03-18T23:08:20.194054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:12.472246+0000","last_clean_scrub_stamp":"2024-03-18T23:07:12.472246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:27:04.482546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:21.156448+0000","last_change":"2024-03-18T23:07:14.068691+0000","last_active":"2024-03-18T23:08:21.156448+0000","last_peered":"2024-03-18T23:08:21.156448+0000","last_clean":"2024-03-18T23:08:21.156448+0000","last_became_active":"2024-03-18T23:07:14.068268+0000","last_became_peered":"2024-03-18T23:07:14.068268+0000","last_unstale":"2024-03-18T23:08:21.156448+0000","last_undegraded":"2024-03-18T23:08:21.156448+0000","last_fullsized":"2024-03-18T23:08:21.156448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:55:56.966209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194938+0000","last_change":"2024-03-18T23:07:14.061079+0000","last_active":"2024-03-18T23:08:20.194938+0000","last_peered":"2024-03-18T23:08:20.194938+0000","last_clean":"2024-03-18T23:08:20.194938+0000","last_became_active":"2024-03-18T23:07:14.060379+0000","last_became_peered":"2024-03-18T23:07:14.060379+0000","last_unstale":"2024-03-18T23:08:20.194938+0000","last_undegraded":"2024-03-18T23:08:20.194938+0000","last_fullsized":"2024-03-18T23:08:20.194938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:17.635874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198584+0000","last_change":"2024-03-18T23:07:14.069349+0000","last_active":"2024-03-18T23:08:20.198584+0000","last_peered":"2024-03-18T23:08:20.198584+0000","last_clean":"2024-03-18T23:08:20.198584+0000","last_became_active":"2024-03-18T23:07:14.068777+0000","last_became_peered":"2024-03-18T23:07:14.068777+0000","last_unstale":"2024-03-18T23:08:20.198584+0000","last_undegraded":"2024-03-18T23:08:20.198584+0000","last_fullsized":"2024-03-18T23:08:20.198584+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:32.509613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198632+0000","last_change":"2024-03-18T23:07:14.062981+0000","last_active":"2024-03-18T23:08:20.198632+0000","last_peered":"2024-03-18T23:08:20.198632+0000","last_clean":"2024-03-18T23:08:20.198632+0000","last_became_active":"2024-03-18T23:07:14.061986+0000","last_became_peered":"2024-03-18T23:07:14.061986+0000","last_unstale":"2024-03-18T23:08:20.198632+0000","last_undegraded":"2024-03-18T23:08:20.198632+0000","last_fullsized":"2024-03-18T23:08:20.198632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:28:46.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.196767+0000","last_change":"2024-03-18T23:07:14.069200+0000","last_active":"2024-03-18T23:08:20.196767+0000","last_peered":"2024-03-18T23:08:20.196767+0000","last_clean":"2024-03-18T23:08:20.196767+0000","last_became_active":"2024-03-18T23:07:14.068893+0000","last_became_peered":"2024-03-18T23:07:14.068893+0000","last_unstale":"2024-03-18T23:08:20.196767+0000","last_undegraded":"2024-03-18T23:08:20.196767+0000","last_fullsized":"2024-03-18T23:08:20.196767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:16:34.488251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.194309+0000","last_change":"2024-03-18T23:07:14.065429+0000","last_active":"2024-03-18T23:08:20.194309+0000","last_peered":"2024-03-18T23:08:20.194309+0000","last_clean":"2024-03-18T23:08:20.194309+0000","last_became_active":"2024-03-18T23:07:14.065083+0000","last_became_peered":"2024-03-18T23:07:14.065083+0000","last_unstale":"2024-03-18T23:08:20.194309+0000","last_undegraded":"2024-03-18T23:08:20.194309+0000","last_fullsized":"2024-03-18T23:08:20.194309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:08:37.317538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195420+0000","last_change":"2024-03-18T23:07:14.059155+0000","last_active":"2024-03-18T23:08:20.195420+0000","last_peered":"2024-03-18T23:08:20.195420+0000","last_clean":"2024-03-18T23:08:20.195420+0000","last_became_active":"2024-03-18T23:07:14.058800+0000","last_became_peered":"2024-03-18T23:07:14.058800+0000","last_unstale":"2024-03-18T23:08:20.195420+0000","last_undegraded":"2024-03-18T23:08:20.195420+0000","last_fullsized":"2024-03-18T23:08:20.195420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:32.994668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.198894+0000","last_change":"2024-03-18T23:07:14.065351+0000","last_active":"2024-03-18T23:08:20.198894+0000","last_peered":"2024-03-18T23:08:20.198894+0000","last_clean":"2024-03-18T23:08:20.198894+0000","last_became_active":"2024-03-18T23:07:14.065012+0000","last_became_peered":"2024-03-18T23:07:14.065012+0000","last_unstale":"2024-03-18T23:08:20.198894+0000","last_undegraded":"2024-03-18T23:08:20.198894+0000","last_fullsized":"2024-03-18T23:08:20.198894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:35.452474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914703+0000","last_change":"2024-03-18T23:07:14.067179+0000","last_active":"2024-03-18T23:08:20.914703+0000","last_peered":"2024-03-18T23:08:20.914703+0000","last_clean":"2024-03-18T23:08:20.914703+0000","last_became_active":"2024-03-18T23:07:14.066899+0000","last_became_peered":"2024-03-18T23:07:14.066899+0000","last_unstale":"2024-03-18T23:08:20.914703+0000","last_undegraded":"2024-03-18T23:08:20.914703+0000","last_fullsized":"2024-03-18T23:08:20.914703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:02.180512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191105+0000","last_change":"2024-03-18T23:07:14.061089+0000","last_active":"2024-03-18T23:08:20.191105+0000","last_peered":"2024-03-18T23:08:20.191105+0000","last_clean":"2024-03-18T23:08:20.191105+0000","last_became_active":"2024-03-18T23:07:14.060787+0000","last_became_peered":"2024-03-18T23:07:14.060787+0000","last_unstale":"2024-03-18T23:08:20.191105+0000","last_undegraded":"2024-03-18T23:08:20.191105+0000","last_fullsized":"2024-03-18T23:08:20.191105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:47:52.029524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.914349+0000","last_change":"2024-03-18T23:07:14.058888+0000","last_active":"2024-03-18T23:08:20.914349+0000","last_peered":"2024-03-18T23:08:20.914349+0000","last_clean":"2024-03-18T23:08:20.914349+0000","last_became_active":"2024-03-18T23:07:14.058576+0000","last_became_peered":"2024-03-18T23:07:14.058576+0000","last_unstale":"2024-03-18T23:08:20.914349+0000","last_undegraded":"2024-03-18T23:08:20.914349+0000","last_fullsized":"2024-03-18T23:08:20.914349+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:27.121268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.191307+0000","last_change":"2024-03-18T23:07:14.076698+0000","last_active":"2024-03-18T23:08:20.191307+0000","last_peered":"2024-03-18T23:08:20.191307+0000","last_clean":"2024-03-18T23:08:20.191307+0000","last_became_active":"2024-03-18T23:07:14.076156+0000","last_became_peered":"2024-03-18T23:07:14.076156+0000","last_unstale":"2024-03-18T23:08:20.191307+0000","last_undegraded":"2024-03-18T23:08:20.191307+0000","last_fullsized":"2024-03-18T23:08:20.191307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:56:01.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.193795+0000","last_change":"2024-03-18T23:07:14.057774+0000","last_active":"2024-03-18T23:08:20.193795+0000","last_peered":"2024-03-18T23:08:20.193795+0000","last_clean":"2024-03-18T23:08:20.193795+0000","last_became_active":"2024-03-18T23:07:14.057456+0000","last_became_peered":"2024-03-18T23:07:14.057456+0000","last_unstale":"2024-03-18T23:08:20.193795+0000","last_undegraded":"2024-03-18T23:08:20.193795+0000","last_fullsized":"2024-03-18T23:08:20.193795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:33.742347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T23:08:20.195066+0000","last_change":"2024-03-18T23:07:14.066689+0000","last_active":"2024-03-18T23:08:20.195066+0000","last_peered":"2024-03-18T23:08:20.195066+0000","last_clean":"2024-03-18T23:08:20.195066+0000","last_became_active":"2024-03-18T23:07:14.066358+0000","last_became_peered":"2024-03-18T23:07:14.066358+0000","last_unstale":"2024-03-18T23:08:20.195066+0000","last_undegraded":"2024-03-18T23:08:20.195066+0000","last_fullsized":"2024-03-18T23:08:20.195066+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T23:07:13.483728+0000","last_clean_scrub_stamp":"2024-03-18T23:07:13.483728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:23:29.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63304,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":447,"num_read_kb":290,"num_write":155,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":604,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285597,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":468891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1119653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":469105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972395,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1061346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":468494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":527199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":469765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":528358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":468805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":469904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1597440,"data_stored":1060639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T23:08:31.789 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-18T23:08:31.789 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T23:08:31.789 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-18T23:08:31.789 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-18T23:08:32.029 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:32.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:32 smithi132 ceph-mon[32020]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 55 B/s, 0 objects/s recovering 2024-03-18T23:08:32.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:32 smithi132 ceph-mon[32020]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:32.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:32 smithi196 ceph-mon[67089]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 55 B/s, 0 objects/s recovering 2024-03-18T23:08:32.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:32 smithi196 ceph-mon[67089]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:32 smithi144 ceph-mon[66781]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 55 B/s, 0 objects/s recovering 2024-03-18T23:08:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:32 smithi144 ceph-mon[66781]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 -- 172.21.15.132:0/1692217590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040102360 msgr2=0x7f30401027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 --2- 172.21.15.132:0/1692217590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040102360 0x7f30401027e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f3028009ca0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 -- 172.21.15.132:0/1692217590 shutdown_connections 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 --2- 172.21.15.132:0/1692217590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 0x7f304010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 --2- 172.21.15.132:0/1692217590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040108410 0x7f30401087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 --2- 172.21.15.132:0/1692217590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040102360 0x7f30401027e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 -- 172.21.15.132:0/1692217590 >> 172.21.15.132:0/1692217590 conn(0x7f3040079440 msgr2=0x7f3040079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 -- 172.21.15.132:0/1692217590 shutdown_connections 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.674+0000 7f3045de9700 1 -- 172.21.15.132:0/1692217590 wait complete. 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 Processor -- start 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 -- start start 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040102360 0x7f304019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 0x7f304019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3040112640 con 0x7f3040108410 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30401124c0 con 0x7f3040102d20 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3045de9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30401127c0 con 0x7f3040102360 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f303ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:33.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f303ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45646/0 (socket says 172.21.15.132:45646) 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f303f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040102360 0x7f304019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f303ffff700 1 -- 172.21.15.132:0/4256254199 learned_addr learned my addr 172.21.15.132:0/4256254199 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.675+0000 7f3036ffd700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 0x7f304019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 -- 172.21.15.132:0/4256254199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040102360 msgr2=0x7f304019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040102360 0x7f304019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 -- 172.21.15.132:0/4256254199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 msgr2=0x7f304019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 0x7f304019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 -- 172.21.15.132:0/4256254199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3040199020 con 0x7f3040108410 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303ffff700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f303800ef30 tx=0x7f303800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30380092e0 con 0x7f3040108410 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3038004bb0 con 0x7f3040108410 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30401992b0 con 0x7f3040108410 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038004280 con 0x7f3040108410 2024-03-18T23:08:33.679 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.676+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30401a55d0 con 0x7f3040108410 2024-03-18T23:08:33.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.677+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3040068fd0 con 0x7f3040108410 2024-03-18T23:08:33.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.678+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f303801b030 con 0x7f3040108410 2024-03-18T23:08:33.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.679+0000 7f303d7fa700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 0x7f302c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:33.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.679+0000 7f303f7fe700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 0x7f302c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:33.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.679+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f30380ce6c0 con 0x7f3040108410 2024-03-18T23:08:33.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.679+0000 7f303f7fe700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 0x7f302c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3040199a00 tx=0x7f3030009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:33.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.683+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3038022020 con 0x7f3040108410 2024-03-18T23:08:33.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.926+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f304002dbb0 con 0x7f3040108410 2024-03-18T23:08:33.927 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.927+0000 7f303d7fa700 1 -- 172.21.15.132:0/4256254199 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3038010040 con 0x7f3040108410 2024-03-18T23:08:33.927 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:33.928 INFO:teuthology.orchestra.run.smithi132.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T23:08:33.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 msgr2=0x7f302c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:33.929 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 0x7f302c0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3040199a00 tx=0x7f3030009260 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 msgr2=0x7f30401988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f303800ef30 tx=0x7f303800c3a0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 shutdown_connections 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f302c0a2920 0x7f302c0a4de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040108410 0x7f30401988c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3040102360 0x7f304019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 --2- 172.21.15.132:0/4256254199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3040102d20 0x7f304019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 >> 172.21.15.132:0/4256254199 conn(0x7f3040079440 msgr2=0x7f3040101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 shutdown_connections 2024-03-18T23:08:33.930 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:33.929+0000 7f3045de9700 1 -- 172.21.15.132:0/4256254199 wait complete. 2024-03-18T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:34 smithi132 ceph-mon[32020]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 50 B/s, 0 objects/s recovering 2024-03-18T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:34 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4256254199' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:08:34.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:34 smithi196 ceph-mon[67089]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 50 B/s, 0 objects/s recovering 2024-03-18T23:08:34.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:34 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4256254199' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:08:34.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:34 smithi144 ceph-mon[66781]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 50 B/s, 0 objects/s recovering 2024-03-18T23:08:34.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:34 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4256254199' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T23:08:34.640 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-18T23:08:34.641 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- 172.21.15.132:0/3702956568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec1022f0 msgr2=0x7f52ec108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3702956568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec1022f0 0x7f52ec108b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- 172.21.15.132:0/3702956568 shutdown_connections 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3702956568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec1022f0 0x7f52ec108b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3702956568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec101930 0x7f52ec101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3702956568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec100740 0x7f52ec100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- 172.21.15.132:0/3702956568 >> 172.21.15.132:0/3702956568 conn(0x7f52ec0fbcb0 msgr2=0x7f52ec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- 172.21.15.132:0/3702956568 shutdown_connections 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- 172.21.15.132:0/3702956568 wait complete. 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 Processor -- start 2024-03-18T23:08:34.748 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.745+0000 7f52f13b2640 1 -- start start 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec100740 0x7f52ec197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10d4e0 con 0x7f52ec101930 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10d360 con 0x7f52ec100740 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10d660 con 0x7f52ec1022f0 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52ea7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52ea7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45654/0 (socket says 172.21.15.132:45654) 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57650/0 (socket says 172.21.15.132:57650) 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52ea7fc640 1 -- 172.21.15.132:0/3324668288 learned_addr learned my addr 172.21.15.132:0/3324668288 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eaffd640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec100740 0x7f52ec197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 -- 172.21.15.132:0/3324668288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec100740 msgr2=0x7f52ec197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:34.749 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec100740 0x7f52ec197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 -- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 msgr2=0x7f52ec198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 -- 172.21.15.132:0/3324668288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec19f730 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52ea7fc640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52eb7fe640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52e402c170 tx=0x7f52e402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4041070 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e4004450 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec19f9c0 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4034440 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.746+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec19fe80 con 0x7f52ec1022f0 2024-03-18T23:08:34.750 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.747+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ec107a60 con 0x7f52ec1022f0 2024-03-18T23:08:34.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.748+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f52e4044070 con 0x7f52ec1022f0 2024-03-18T23:08:34.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.749+0000 7f52d3fff640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 0x7f52d40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:34.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.749+0000 7f52eaffd640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 0x7f52d40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:34.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.749+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f52e40f07f0 con 0x7f52ec1022f0 2024-03-18T23:08:34.751 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.749+0000 7f52eaffd640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 0x7f52d40a4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52ec101790 tx=0x7f52dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:34.753 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.753+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f52e40bd200 con 0x7f52ec1022f0 2024-03-18T23:08:34.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.931+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52ec062420 con 0x7f52ec1022f0 2024-03-18T23:08:34.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.934+0000 7f52d3fff640 1 -- 172.21.15.132:0/3324668288 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7f52e40c12a0 con 0x7f52ec1022f0 2024-03-18T23:08:34.934 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:34.934 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:08:34.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 msgr2=0x7f52d40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:34.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 0x7f52d40a4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52ec101790 tx=0x7f52dc009500 comp rx=0 tx=0).stop 2024-03-18T23:08:34.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 msgr2=0x7f52ec19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:34.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52e402c170 tx=0x7f52e402c6c0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 shutdown_connections 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52d40a2890 0x7f52d40a4d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52ec1022f0 0x7f52ec19f0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52ec101930 0x7f52ec198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 --2- 172.21.15.132:0/3324668288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52ec100740 0x7f52ec197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 >> 172.21.15.132:0/3324668288 conn(0x7f52ec0fbcb0 msgr2=0x7f52ec104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 shutdown_connections 2024-03-18T23:08:34.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:34.935+0000 7f52f13b2640 1 -- 172.21.15.132:0/3324668288 wait complete. 2024-03-18T23:08:34.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:08:34.948 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- 172.21.15.132:0/2071258741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c1022c0 msgr2=0x7f433c1046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- 172.21.15.132:0/2071258741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c1022c0 0x7f433c1046b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4330009900 tx=0x7f433002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- 172.21.15.132:0/2071258741 shutdown_connections 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- 172.21.15.132:0/2071258741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c107520 0x7f433c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- 172.21.15.132:0/2071258741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c104bf0 0x7f433c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- 172.21.15.132:0/2071258741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c1022c0 0x7f433c1046b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- 172.21.15.132:0/2071258741 >> 172.21.15.132:0/2071258741 conn(0x7f433c0fbcd0 msgr2=0x7f433c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- 172.21.15.132:0/2071258741 shutdown_connections 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- 172.21.15.132:0/2071258741 wait complete. 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 Processor -- start 2024-03-18T23:08:35.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- start start 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c1022c0 0x7f433c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 0x7f433c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d4f0 con 0x7f433c104bf0 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d370 con 0x7f433c1022c0 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4342e44640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d670 con 0x7f433c107520 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f433bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f433bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45676/0 (socket says 172.21.15.132:45676) 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f433bfff640 1 -- 172.21.15.132:0/1689061737 learned_addr learned my addr 172.21.15.132:0/1689061737 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.053+0000 7f4340be0640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c1022c0 0x7f433c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f43413e1640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 0x7f433c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 -- 172.21.15.132:0/1689061737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 msgr2=0x7f433c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 0x7f433c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 -- 172.21.15.132:0/1689061737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c1022c0 msgr2=0x7f433c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c1022c0 0x7f433c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 -- 172.21.15.132:0/1689061737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c19f7b0 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f43413e1640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 0x7f433c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f433bfff640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432800ccc0 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4328005500 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c19faa0 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328010690 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.054+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c19ffa0 con 0x7f433c104bf0 2024-03-18T23:08:35.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.055+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c0fd7c0 con 0x7f433c104bf0 2024-03-18T23:08:35.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.056+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f43280046c0 con 0x7f433c104bf0 2024-03-18T23:08:35.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.057+0000 7f4339ffb640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 0x7f432c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.057+0000 7f4340be0640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 0x7f432c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.057+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f43280cefd0 con 0x7f433c104bf0 2024-03-18T23:08:35.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.057+0000 7f4340be0640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 0x7f432c0a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f433002c7b0 tx=0x7f4330005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.061+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f432809bf20 con 0x7f433c104bf0 2024-03-18T23:08:35.237 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.237+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f433c02d440 con 0x7f433c104bf0 2024-03-18T23:08:35.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.237+0000 7f4339ffb640 1 -- 172.21.15.132:0/1689061737 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f432809b820 con 0x7f433c104bf0 2024-03-18T23:08:35.238 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 msgr2=0x7f432c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 0x7f432c0a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f433002c7b0 tx=0x7f4330005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 msgr2=0x7f433c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f432800e9f0 tx=0x7f432800eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 shutdown_connections 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f432c0a2890 0x7f432c0a4d50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f433c107520 0x7f433c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f433c104bf0 0x7f433c1980d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 --2- 172.21.15.132:0/1689061737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f433c1022c0 0x7f433c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 >> 172.21.15.132:0/1689061737 conn(0x7f433c0fbcd0 msgr2=0x7f433c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.240 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 shutdown_connections 2024-03-18T23:08:35.241 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.239+0000 7f4342e44640 1 -- 172.21.15.132:0/1689061737 wait complete. 2024-03-18T23:08:35.250 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:35 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3324668288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:35.251 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:08:35.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/3671156679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 msgr2=0x7f2b28108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/3671156679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b28108b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009900 tx=0x7f2b2002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/3671156679 shutdown_connections 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/3671156679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b28108b60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/3671156679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28101910 0x7f2b28101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/3671156679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28100720 0x7f2b28100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/3671156679 >> 172.21.15.132:0/3671156679 conn(0x7f2b280fbcb0 msgr2=0x7f2b280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/3671156679 shutdown_connections 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.354+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/3671156679 wait complete. 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 Processor -- start 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 -- start start 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28101910 0x7f2b28198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b2819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810d4c0 con 0x7f2b281022d0 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810d340 con 0x7f2b28101910 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b2d7d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2810d640 con 0x7f2b28100720 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b277fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b2819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57700/0 (socket says 172.21.15.132:57700) 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 -- 172.21.15.132:0/4289442264 learned_addr learned my addr 172.21.15.132:0/4289442264 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b267fc640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28101910 0x7f2b28198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 -- 172.21.15.132:0/4289442264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28101910 msgr2=0x7f2b28198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28101910 0x7f2b28198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 -- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 msgr2=0x7f2b2819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b2819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.355+0000 7f2b26ffd640 1 -- 172.21.15.132:0/4289442264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2819f730 con 0x7f2b28100720 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b26ffd640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800ea80 tx=0x7f2b1800ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800ce40 con 0x7f2b28100720 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2819fa20 con 0x7f2b28100720 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18011950 con 0x7f2b28100720 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1801ab80 con 0x7f2b28100720 2024-03-18T23:08:35.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.356+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2819ffc0 con 0x7f2b28100720 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.357+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b28107a40 con 0x7f2b28100720 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.358+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f2b18011b00 con 0x7f2b28100720 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.358+0000 7f2b0ffff640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.358+0000 7f2b267fc640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.358+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2b180cf1b0 con 0x7f2b28100720 2024-03-18T23:08:35.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.358+0000 7f2b267fc640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 0x7f2b100a4d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b28199020 tx=0x7f2b1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.363+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b18016030 con 0x7f2b28100720 2024-03-18T23:08:35.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:35 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3324668288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:35.538 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.538+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b2802d1f0 con 0x7f2b28100720 2024-03-18T23:08:35.539 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.538+0000 7f2b0ffff640 1 -- 172.21.15.132:0/4289442264 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f2b1809c100 con 0x7f2b28100720 2024-03-18T23:08:35.539 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 msgr2=0x7f2b100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 0x7f2b100a4d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b28199020 tx=0x7f2b1c005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 msgr2=0x7f2b28197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800ea80 tx=0x7f2b1800ef40 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 shutdown_connections 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b100a2890 0x7f2b100a4d50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b281022d0 0x7f2b2819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b28101910 0x7f2b28198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 --2- 172.21.15.132:0/4289442264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b28100720 0x7f2b28197b10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 >> 172.21.15.132:0/4289442264 conn(0x7f2b280fbcb0 msgr2=0x7f2b28104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 shutdown_connections 2024-03-18T23:08:35.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.541+0000 7f2b2d7d4640 1 -- 172.21.15.132:0/4289442264 wait complete. 2024-03-18T23:08:35.553 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:08:35.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:35 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3324668288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:35.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 -- 172.21.15.132:0/3220303829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64401022e0 msgr2=0x7f6440108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 --2- 172.21.15.132:0/3220303829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64401022e0 0x7f6440108b70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6438009ca0 tx=0x7f643802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 -- 172.21.15.132:0/3220303829 shutdown_connections 2024-03-18T23:08:35.660 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 --2- 172.21.15.132:0/3220303829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64401022e0 0x7f6440108b70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 --2- 172.21.15.132:0/3220303829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6440101920 0x7f6440101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 --2- 172.21.15.132:0/3220303829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 -- 172.21.15.132:0/3220303829 >> 172.21.15.132:0/3220303829 conn(0x7f64400fbcc0 msgr2=0x7f64400fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 -- 172.21.15.132:0/3220303829 shutdown_connections 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.657+0000 7f6447ec6640 1 -- 172.21.15.132:0/3220303829 wait complete. 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 Processor -- start 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 -- start start 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64401022e0 0x7f6440103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c3b0 con 0x7f6440101920 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c230 con 0x7f6440100730 2024-03-18T23:08:35.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6447ec6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c530 con 0x7f64401022e0 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45698/0 (socket says 172.21.15.132:45698) 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 -- 172.21.15.132:0/2646991809 learned_addr learned my addr 172.21.15.132:0/2646991809 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6446463640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64401022e0 0x7f6440103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445c62640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 -- 172.21.15.132:0/2646991809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64401022e0 msgr2=0x7f6440103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64401022e0 0x7f6440103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 -- 172.21.15.132:0/2646991809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 msgr2=0x7f6440104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445461640 1 -- 172.21.15.132:0/2646991809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6440103ba0 con 0x7f6440101920 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.658+0000 7f6445c62640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:35.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6445461640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f643000e9f0 tx=0x7f643000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643000cd80 con 0x7f6440101920 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64401a1c50 con 0x7f6440101920 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6430005500 con 0x7f6440101920 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430010690 con 0x7f6440101920 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.659+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64401a21a0 con 0x7f6440101920 2024-03-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.660+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64401a23d0 con 0x7f6440101920 2024-03-18T23:08:35.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.661+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f64300046c0 con 0x7f6440101920 2024-03-18T23:08:35.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.662+0000 7f6436ffd640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 0x7f642c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.662+0000 7f6445c62640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 0x7f642c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.662+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f64300cf310 con 0x7f6440101920 2024-03-18T23:08:35.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.662+0000 7f6445c62640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 0x7f642c0a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6440101780 tx=0x7f643c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.665+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6430053740 con 0x7f6440101920 2024-03-18T23:08:35.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.840+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64401a2760 con 0x7f6440101920 2024-03-18T23:08:35.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.840+0000 7f6436ffd640 1 -- 172.21.15.132:0/2646991809 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f643009c200 con 0x7f6440101920 2024-03-18T23:08:35.841 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:08:35.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 msgr2=0x7f642c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 0x7f642c0a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6440101780 tx=0x7f643c005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:35.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 msgr2=0x7f6440102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f643000e9f0 tx=0x7f643000edf0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 shutdown_connections 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f642c0a2890 0x7f642c0a4d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64401022e0 0x7f6440103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6440101920 0x7f6440102dd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 --2- 172.21.15.132:0/2646991809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6440100730 0x7f6440104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.842+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 >> 172.21.15.132:0/2646991809 conn(0x7f64400fbcc0 msgr2=0x7f6440069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.843+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 shutdown_connections 2024-03-18T23:08:35.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.843+0000 7f6447ec6640 1 -- 172.21.15.132:0/2646991809 wait complete. 2024-03-18T23:08:35.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- 172.21.15.132:0/2607501174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 msgr2=0x7f0368108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 --2- 172.21.15.132:0/2607501174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f0368108c00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0360009ca0 tx=0x7f036002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- 172.21.15.132:0/2607501174 shutdown_connections 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 --2- 172.21.15.132:0/2607501174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f0368108c00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 --2- 172.21.15.132:0/2607501174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681019b0 0x7f0368101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 --2- 172.21.15.132:0/2607501174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681007c0 0x7f0368100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- 172.21.15.132:0/2607501174 >> 172.21.15.132:0/2607501174 conn(0x7f03680fbd50 msgr2=0x7f03680fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- 172.21.15.132:0/2607501174 shutdown_connections 2024-03-18T23:08:35.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- 172.21.15.132:0/2607501174 wait complete. 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 Processor -- start 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.956+0000 7f036ee36640 1 -- start start 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 0x7f036806a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681019b0 0x7f036819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f036810c3b0 con 0x7f0368102370 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f036810c230 con 0x7f03681007c0 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f036810c530 con 0x7f03681019b0 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036cbd2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 0x7f036806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45718/0 (socket says 172.21.15.132:45718) 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 -- 172.21.15.132:0/3563437354 learned_addr learned my addr 172.21.15.132:0/3563437354 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f0367fff640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681019b0 0x7f036819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 -- 172.21.15.132:0/3563437354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681019b0 msgr2=0x7f036819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681019b0 0x7f036819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 -- 172.21.15.132:0/3563437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 msgr2=0x7f036806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 0x7f036806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 -- 172.21.15.132:0/3563437354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036819f710 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036cbd2640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 0x7f036806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036d3d3640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0360037040 tx=0x7f0360032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360041020 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0360034cd0 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036819f940 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.957+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360039040 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.958+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036819fe80 con 0x7f0368102370 2024-03-18T23:08:35.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.958+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368107a80 con 0x7f0368102370 2024-03-18T23:08:35.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.960+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f0360044020 con 0x7f0368102370 2024-03-18T23:08:35.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.960+0000 7f0365ffb640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 0x7f03580a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:35.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.960+0000 7f036cbd2640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 0x7f03580a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:35.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.960+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f03600f08f0 con 0x7f0368102370 2024-03-18T23:08:35.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.961+0000 7f036cbd2640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 0x7f03580a4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0368101810 tx=0x7f035c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:35.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:35.965+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f036004b020 con 0x7f0368102370 2024-03-18T23:08:36.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.178+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f036802d3d0 con 0x7f0368102370 2024-03-18T23:08:36.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.179+0000 7f0365ffb640 1 -- 172.21.15.132:0/3563437354 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7f03600bd9e0 con 0x7f0368102370 2024-03-18T23:08:36.180 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:36.180 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:08:36.180 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 msgr2=0x7f03580a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 0x7f03580a4dd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0368101810 tx=0x7f035c005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 msgr2=0x7f036819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0360037040 tx=0x7f0360032f40 comp rx=0 tx=0).stop 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 shutdown_connections 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f03580a2910 0x7f03580a4dd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0368102370 0x7f036819ee00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03681019b0 0x7f036819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 --2- 172.21.15.132:0/3563437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03681007c0 0x7f036806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 >> 172.21.15.132:0/3563437354 conn(0x7f03680fbd50 msgr2=0x7f0368104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 shutdown_connections 2024-03-18T23:08:36.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.181+0000 7f036ee36640 1 -- 172.21.15.132:0/3563437354 wait complete. 2024-03-18T23:08:36.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:36.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.294+0000 7fe7f4273640 1 -- 172.21.15.132:0/3262839654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec101930 msgr2=0x7fe7ec101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.294+0000 7fe7f4273640 1 --2- 172.21.15.132:0/3262839654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec101930 0x7fe7ec101db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009900 tx=0x7fe7e802bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- 172.21.15.132:0/3262839654 shutdown_connections 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- 172.21.15.132:0/3262839654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 0x7fe7ec108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- 172.21.15.132:0/3262839654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec101930 0x7fe7ec101db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- 172.21.15.132:0/3262839654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec100740 0x7fe7ec100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- 172.21.15.132:0/3262839654 >> 172.21.15.132:0/3262839654 conn(0x7fe7ec0fbcb0 msgr2=0x7fe7ec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- 172.21.15.132:0/3262839654 shutdown_connections 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- 172.21.15.132:0/3262839654 wait complete. 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 Processor -- start 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- start start 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 0x7fe7ec1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.298 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 0x7fe7ec19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10d4e0 con 0x7fe7ec100740 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10d360 con 0x7fe7ec101930 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f4273640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10d660 con 0x7fe7ec1022f0 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f200f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f200f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45738/0 (socket says 172.21.15.132:45738) 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f200f640 1 -- 172.21.15.132:0/2022318820 learned_addr learned my addr 172.21.15.132:0/2022318820 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f180e640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 0x7fe7ec1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.295+0000 7fe7f2810640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 0x7fe7ec19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 -- 172.21.15.132:0/2022318820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 msgr2=0x7fe7ec19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 0x7fe7ec19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 -- 172.21.15.132:0/2022318820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 msgr2=0x7fe7ec1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 0x7fe7ec1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 -- 172.21.15.132:0/2022318820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19f780 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f180e640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 0x7fe7ec1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f200f640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9c0 tx=0x7fe7dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00cd80 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc0035f0 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19fa70 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc010690 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.296+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec19ff90 con 0x7fe7ec100740 2024-03-18T23:08:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.297+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec107a60 con 0x7fe7ec100740 2024-03-18T23:08:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.298+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fe7dc0037a0 con 0x7fe7ec100740 2024-03-18T23:08:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.298+0000 7fe7e37fe640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 0x7fe7d80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.299+0000 7fe7f180e640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 0x7fe7d80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.299+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe7dc0cef90 con 0x7fe7ec100740 2024-03-18T23:08:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.299+0000 7fe7f180e640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 0x7fe7d80a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7e802c7b0 tx=0x7fe7e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.303+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe7dc09bee0 con 0x7fe7ec100740 2024-03-18T23:08:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:36 smithi144 ceph-mon[66781]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:36 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1689061737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:36 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/4289442264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.359 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:36 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2646991809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:36 smithi196 ceph-mon[67089]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:36 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1689061737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:36 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/4289442264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:36 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2646991809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.478+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7ec02d1f0 con 0x7fe7ec100740 2024-03-18T23:08:36.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.479+0000 7fe7e37fe640 1 -- 172.21.15.132:0/2022318820 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7fe7dc09b7e0 con 0x7fe7ec100740 2024-03-18T23:08:36.480 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:36.480 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:08:36.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 msgr2=0x7fe7d80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 0x7fe7d80a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7e802c7b0 tx=0x7fe7e8005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 msgr2=0x7fe7ec197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9c0 tx=0x7fe7dc00ee80 comp rx=0 tx=0).stop 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 shutdown_connections 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fe7d80a2890 0x7fe7d80a4d50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.481+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7ec1022f0 0x7fe7ec19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.482+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7ec101930 0x7fe7ec1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.482+0000 7fe7f4273640 1 --2- 172.21.15.132:0/2022318820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ec100740 0x7fe7ec197b60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.482+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 >> 172.21.15.132:0/2022318820 conn(0x7fe7ec0fbcb0 msgr2=0x7fe7ec104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.482+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 shutdown_connections 2024-03-18T23:08:36.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.482+0000 7fe7f4273640 1 -- 172.21.15.132:0/2022318820 wait complete. 2024-03-18T23:08:36.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:36 smithi132 ceph-mon[32020]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:36.492 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:36 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1689061737' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:36 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/4289442264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.493 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:36 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2646991809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:36.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:08:36.494 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 -- 172.21.15.132:0/4142245588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64100720 msgr2=0x7f2b64100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 --2- 172.21.15.132:0/4142245588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64100720 0x7f2b64100b40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009ca0 tx=0x7f2b6002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 -- 172.21.15.132:0/4142245588 shutdown_connections 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 --2- 172.21.15.132:0/4142245588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b64108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 --2- 172.21.15.132:0/4142245588 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64101910 0x7f2b64101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 --2- 172.21.15.132:0/4142245588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64100720 0x7f2b64100b40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.599+0000 7f2b6b537640 1 -- 172.21.15.132:0/4142245588 >> 172.21.15.132:0/4142245588 conn(0x7f2b640fbcb0 msgr2=0x7f2b640fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- 172.21.15.132:0/4142245588 shutdown_connections 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- 172.21.15.132:0/4142245588 wait complete. 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 Processor -- start 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- start start 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 0x7f2b64196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b6419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410c370 con 0x7f2b64101910 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410c1f0 con 0x7f2b641022d0 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b6b537640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410c4f0 con 0x7f2b64100720 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b692d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b68ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 0x7f2b64196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b692d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57804/0 (socket says 172.21.15.132:57804) 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b692d3640 1 -- 172.21.15.132:0/2382146858 learned_addr learned my addr 172.21.15.132:0/2382146858 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.600+0000 7f2b69ad4640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b6419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 -- 172.21.15.132:0/2382146858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 msgr2=0x7f2b64196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 -- 172.21.15.132:0/2382146858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 msgr2=0x7f2b6419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b6419e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 -- 172.21.15.132:0/2382146858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6419e690 con 0x7f2b64101910 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b692d3640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b69ad4640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b6419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b68ad2640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 0x7f2b64196fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9f0 tx=0x7f2b5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400cd80 con 0x7f2b64101910 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b640689e0 con 0x7f2b64101910 2024-03-18T23:08:36.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b540035f0 con 0x7f2b64101910 2024-03-18T23:08:36.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54010670 con 0x7f2b64101910 2024-03-18T23:08:36.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.601+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b64068f80 con 0x7f2b64101910 2024-03-18T23:08:36.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.602+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b64107580 con 0x7f2b64101910 2024-03-18T23:08:36.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.603+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f2b540037a0 con 0x7f2b64101910 2024-03-18T23:08:36.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.603+0000 7f2b5a7fc640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 0x7f2b500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.604+0000 7f2b692d3640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 0x7f2b500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.604+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2b540cee80 con 0x7f2b64101910 2024-03-18T23:08:36.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.604+0000 7f2b692d3640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 0x7f2b500a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b60032420 tx=0x7f2b60032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.608+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b5409bdd0 con 0x7f2b64101910 2024-03-18T23:08:36.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.781+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b64107950 con 0x7f2b64101910 2024-03-18T23:08:36.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.782+0000 7f2b5a7fc640 1 -- 172.21.15.132:0/2382146858 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f2b5409b6d0 con 0x7f2b64101910 2024-03-18T23:08:36.783 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:08:36.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 msgr2=0x7f2b500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 0x7f2b500a4d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b60032420 tx=0x7f2b60032590 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 msgr2=0x7f2b64196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 0x7f2b64196fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9f0 tx=0x7f2b5400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 shutdown_connections 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f2b500a2890 0x7f2b500a4d50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b641022d0 0x7f2b6419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b64101910 0x7f2b64196fb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 --2- 172.21.15.132:0/2382146858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b64100720 0x7f2b64196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 >> 172.21.15.132:0/2382146858 conn(0x7f2b640fbcb0 msgr2=0x7f2b64104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 shutdown_connections 2024-03-18T23:08:36.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.784+0000 7f2b6b537640 1 -- 172.21.15.132:0/2382146858 wait complete. 2024-03-18T23:08:36.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1950938574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d0100720 msgr2=0x7ff3d0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1950938574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d0100720 0x7ff3d0100b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1950938574 shutdown_connections 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1950938574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d01022d0 0x7ff3d0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1950938574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1950938574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d0100720 0x7ff3d0100b40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1950938574 >> 172.21.15.132:0/1950938574 conn(0x7ff3d00fbcb0 msgr2=0x7ff3d00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1950938574 shutdown_connections 2024-03-18T23:08:36.903 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.900+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1950938574 wait complete. 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 Processor -- start 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 -- start start 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0100720 0x7ff3d0104820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c370 con 0x7ff3d01022d0 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c1f0 con 0x7ff3d0101910 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3d4fd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c4f0 con 0x7ff3d0100720 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45758/0 (socket says 172.21.15.132:45758) 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 -- 172.21.15.132:0/1075355494 learned_addr learned my addr 172.21.15.132:0/1075355494 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ce59c640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0100720 0x7ff3d0104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3c7fff640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 -- 172.21.15.132:0/1075355494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0100720 msgr2=0x7ff3d0104820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0100720 0x7ff3d0104820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.901+0000 7ff3ced9d640 1 -- 172.21.15.132:0/1075355494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 msgr2=0x7ff3d0102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3ced9d640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3ced9d640 1 -- 172.21.15.132:0/1075355494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d0103c30 con 0x7ff3d01022d0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3c7fff640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3ced9d640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800eb10 tx=0x7ff3c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8004b20 con 0x7ff3d01022d0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8004cc0 con 0x7ff3d01022d0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01a1c70 con 0x7ff3d01022d0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8010760 con 0x7ff3d01022d0 2024-03-18T23:08:36.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.902+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a2170 con 0x7ff3d01022d0 2024-03-18T23:08:36.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.903+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d01a23a0 con 0x7ff3d01022d0 2024-03-18T23:08:36.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.904+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7ff3c8005020 con 0x7ff3d01022d0 2024-03-18T23:08:36.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.905+0000 7ff3c77fe640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 0x7ff3bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:36.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.905+0000 7ff3ce59c640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 0x7ff3bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:36.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.905+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff3c80cf1c0 con 0x7ff3d01022d0 2024-03-18T23:08:36.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.905+0000 7ff3ce59c640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 0x7ff3bc0a4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:36.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:36.909+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3c809c110 con 0x7ff3d01022d0 2024-03-18T23:08:37.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.085+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3d01a2570 con 0x7ff3d01022d0 2024-03-18T23:08:37.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.085+0000 7ff3c77fe640 1 -- 172.21.15.132:0/1075355494 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7ff3c809ba10 con 0x7ff3d01022d0 2024-03-18T23:08:37.086 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 msgr2=0x7ff3bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 0x7ff3bc0a4d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 msgr2=0x7ff3d01033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800eb10 tx=0x7ff3c800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 shutdown_connections 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7ff3bc0a2890 0x7ff3bc0a4d50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3d01022d0 0x7ff3d01033a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0101910 0x7ff3d0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 --2- 172.21.15.132:0/1075355494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d0100720 0x7ff3d0104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 >> 172.21.15.132:0/1075355494 conn(0x7ff3d00fbcb0 msgr2=0x7ff3d0069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.088 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 shutdown_connections 2024-03-18T23:08:37.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.087+0000 7ff3d4fd4640 1 -- 172.21.15.132:0/1075355494 wait complete. 2024-03-18T23:08:37.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:08:37.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- 172.21.15.132:0/1271938330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2384102a20 msgr2=0x7f2384102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 --2- 172.21.15.132:0/1271938330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2384102a20 0x7f2384102ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2374009900 tx=0x7f237402bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- 172.21.15.132:0/1271938330 shutdown_connections 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 --2- 172.21.15.132:0/1271938330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23841033e0 0x7f2384109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 --2- 172.21.15.132:0/1271938330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2384102a20 0x7f2384102ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 --2- 172.21.15.132:0/1271938330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384101830 0x7f2384101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- 172.21.15.132:0/1271938330 >> 172.21.15.132:0/1271938330 conn(0x7f23840fcdc0 msgr2=0x7f23840ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- 172.21.15.132:0/1271938330 shutdown_connections 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- 172.21.15.132:0/1271938330 wait complete. 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 Processor -- start 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.205+0000 7f238c790640 1 -- start start 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384102a20 0x7f2384198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23841033e0 0x7f238419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f238410d4b0 con 0x7f2384101830 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f238410d330 con 0x7f2384102a20 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f238410d630 con 0x7f23841033e0 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45784/0 (socket says 172.21.15.132:45784) 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 -- 172.21.15.132:0/205748866 learned_addr learned my addr 172.21.15.132:0/205748866 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f2389d2b640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384102a20 0x7f2384198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238ad2d640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23841033e0 0x7f238419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 -- 172.21.15.132:0/205748866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23841033e0 msgr2=0x7f238419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23841033e0 0x7f238419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 -- 172.21.15.132:0/205748866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384102a20 msgr2=0x7f2384198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384102a20 0x7f2384198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 -- 172.21.15.132:0/205748866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238419f770 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238a52c640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9c0 tx=0x7f238000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000cd80 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238419fa60 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380010690 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.206+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238419ff80 con 0x7f2384101830 2024-03-18T23:08:37.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.207+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2384108c20 con 0x7f2384101830 2024-03-18T23:08:37.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.208+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f23800037a0 con 0x7f2384101830 2024-03-18T23:08:37.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.209+0000 7f237b7fe640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 0x7f23700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.209+0000 7f2389d2b640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 0x7f23700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.209+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f23800cef40 con 0x7f2384101830 2024-03-18T23:08:37.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.209+0000 7f2389d2b640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 0x7f23700a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f237402c7b0 tx=0x7f2374005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.213+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f238009be90 con 0x7f2384101830 2024-03-18T23:08:37.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3563437354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2022318820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2382146858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1075355494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.392 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.392+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f238402d1f0 con 0x7f2384101830 2024-03-18T23:08:37.393 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.392+0000 7f237b7fe640 1 -- 172.21.15.132:0/205748866 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f238009b790 con 0x7f2384101830 2024-03-18T23:08:37.393 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 msgr2=0x7f23700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 0x7f23700a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f237402c7b0 tx=0x7f2374005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 msgr2=0x7f2384197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f238000e9c0 tx=0x7f238000ee80 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 shutdown_connections 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f23700a2890 0x7f23700a4d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23841033e0 0x7f238419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2384102a20 0x7f2384198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 --2- 172.21.15.132:0/205748866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2384101830 0x7f2384197b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 >> 172.21.15.132:0/205748866 conn(0x7f23840fcdc0 msgr2=0x7f2384105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 shutdown_connections 2024-03-18T23:08:37.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.394+0000 7f238c790640 1 -- 172.21.15.132:0/205748866 wait complete. 2024-03-18T23:08:37.406 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:08:37.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3563437354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2022318820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2382146858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1075355494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- 172.21.15.132:0/1954308227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d01022f0 msgr2=0x7f21d0108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/1954308227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d01022f0 0x7f21d0108b80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009ca0 tx=0x7f21c802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- 172.21.15.132:0/1954308227 shutdown_connections 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/1954308227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d01022f0 0x7f21d0108b80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/1954308227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0101930 0x7f21d0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/1954308227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d0100740 0x7f21d0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- 172.21.15.132:0/1954308227 >> 172.21.15.132:0/1954308227 conn(0x7f21d00fbcb0 msgr2=0x7f21d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- 172.21.15.132:0/1954308227 shutdown_connections 2024-03-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- 172.21.15.132:0/1954308227 wait complete. 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 Processor -- start 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- start start 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 0x7f21d0069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d01022f0 0x7f21d019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21d010c3a0 con 0x7f21d0101930 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21d010c220 con 0x7f21d0100740 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.507+0000 7f21d7ba9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21d010c520 con 0x7f21d01022f0 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45800/0 (socket says 172.21.15.132:45800) 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 -- 172.21.15.132:0/3052205277 learned_addr learned my addr 172.21.15.132:0/3052205277 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5945640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 0x7f21d0069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d6146640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d01022f0 0x7f21d019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 -- 172.21.15.132:0/3052205277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d01022f0 msgr2=0x7f21d019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d01022f0 0x7f21d019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 -- 172.21.15.132:0/3052205277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 msgr2=0x7f21d0069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 0x7f21d0069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 -- 172.21.15.132:0/3052205277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d019f710 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5945640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 0x7f21d0069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d5144640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ea60 tx=0x7f21cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00cde0 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21cc0035f0 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d019fa00 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc010670 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.508+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d019ff50 con 0x7f21d0101930 2024-03-18T23:08:37.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.509+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d0107a60 con 0x7f21d0101930 2024-03-18T23:08:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.510+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f21cc0037a0 con 0x7f21d0101930 2024-03-18T23:08:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.511+0000 7f21c6ffd640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 0x7f21bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.511+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f21cc0cef90 con 0x7f21d0101930 2024-03-18T23:08:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.511+0000 7f21d5945640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 0x7f21bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.511+0000 7f21d5945640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 0x7f21bc0a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21d0101790 tx=0x7f21c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.514+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21cc09bf70 con 0x7f21d0101930 2024-03-18T23:08:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3563437354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2022318820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2382146858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1075355494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:37.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.725+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21d002d1f0 con 0x7f21d0101930 2024-03-18T23:08:37.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.726+0000 7f21c6ffd640 1 -- 172.21.15.132:0/3052205277 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7f21cc09b870 con 0x7f21d0101930 2024-03-18T23:08:37.727 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:37.727 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:08:37.727 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 msgr2=0x7f21bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 0x7f21bc0a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21d0101790 tx=0x7f21c0005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 msgr2=0x7f21d019c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00ea60 tx=0x7f21cc00ef20 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 shutdown_connections 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f21bc0a2890 0x7f21bc0a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21d01022f0 0x7f21d019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21d0101930 0x7f21d019c810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 --2- 172.21.15.132:0/3052205277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d0100740 0x7f21d0069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 >> 172.21.15.132:0/3052205277 conn(0x7f21d00fbcb0 msgr2=0x7f21d0104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 shutdown_connections 2024-03-18T23:08:37.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.729+0000 7f21d7ba9640 1 -- 172.21.15.132:0/3052205277 wait complete. 2024-03-18T23:08:37.740 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:37.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.840+0000 7f963f1a9640 1 -- 172.21.15.132:0/804337190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 msgr2=0x7f96381002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.840+0000 7f963f1a9640 1 --2- 172.21.15.132:0/804337190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f96381002a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f962c009900 tx=0x7f962c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- 172.21.15.132:0/804337190 shutdown_connections 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- 172.21.15.132:0/804337190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101a20 0x7f9638108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- 172.21.15.132:0/804337190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101060 0x7f96381014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- 172.21.15.132:0/804337190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f96381002a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- 172.21.15.132:0/804337190 >> 172.21.15.132:0/804337190 conn(0x7f96380fb440 msgr2=0x7f96380fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- 172.21.15.132:0/804337190 shutdown_connections 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- 172.21.15.132:0/804337190 wait complete. 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 Processor -- start 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- start start 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101060 0x7f9638198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101a20 0x7f963819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f963810d4d0 con 0x7f96380ffe80 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f963810d350 con 0x7f9638101a20 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963f1a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f963810d650 con 0x7f9638101060 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963cf45640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963cf45640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45812/0 (socket says 172.21.15.132:45812) 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963cf45640 1 -- 172.21.15.132:0/3823655782 learned_addr learned my addr 172.21.15.132:0/3823655782 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f9637fff640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101060 0x7f9638198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.841+0000 7f963d746640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101a20 0x7f963819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 -- 172.21.15.132:0/3823655782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101060 msgr2=0x7f9638198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101060 0x7f9638198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 -- 172.21.15.132:0/3823655782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101a20 msgr2=0x7f963819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:37.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101a20 0x7f963819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 -- 172.21.15.132:0/3823655782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963819f760 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963cf45640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f962c000c00 tx=0x7f962c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c041070 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963819f9f0 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f962c004450 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c034440 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.842+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963819fee0 con 0x7f96380ffe80 2024-03-18T23:08:37.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.843+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9638107220 con 0x7f96380ffe80 2024-03-18T23:08:37.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.844+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f962c044070 con 0x7f96380ffe80 2024-03-18T23:08:37.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.845+0000 7f9635ffb640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 0x7f96280a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:37.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.845+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f962c0f0590 con 0x7f96380ffe80 2024-03-18T23:08:37.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.845+0000 7f9637fff640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 0x7f96280a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:37.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.845+0000 7f9637fff640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 0x7f96280a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f96240095f0 tx=0x7f9624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:37.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:37.849+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f962c0bd030 con 0x7f96380ffe80 2024-03-18T23:08:38.024 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.024+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9638062450 con 0x7f96380ffe80 2024-03-18T23:08:38.026 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.025+0000 7f9635ffb640 1 -- 172.21.15.132:0/3823655782 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7f962c0c10d0 con 0x7f96380ffe80 2024-03-18T23:08:38.026 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:38.026 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:08:38.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.027+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 msgr2=0x7f96280a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.027+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 0x7f96280a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f96240095f0 tx=0x7f9624005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:38.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.027+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 msgr2=0x7f9638197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.028 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.027+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f962c000c00 tx=0x7f962c02c640 comp rx=0 tx=0).stop 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 shutdown_connections 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f96280a2890 0x7f96280a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9638101a20 0x7f963819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9638101060 0x7f9638198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 --2- 172.21.15.132:0/3823655782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96380ffe80 0x7f9638197b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 >> 172.21.15.132:0/3823655782 conn(0x7f96380fb440 msgr2=0x7f96381043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 shutdown_connections 2024-03-18T23:08:38.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.028+0000 7f963f1a9640 1 -- 172.21.15.132:0/3823655782 wait complete. 2024-03-18T23:08:38.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:08:38.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- 172.21.15.132:0/2256220122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0104c70 msgr2=0x7f54f0107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- 172.21.15.132:0/2256220122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0104c70 0x7f54f0107060 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009900 tx=0x7f54e002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:38.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- 172.21.15.132:0/2256220122 shutdown_connections 2024-03-18T23:08:38.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- 172.21.15.132:0/2256220122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 0x7f54f0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- 172.21.15.132:0/2256220122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0104c70 0x7f54f0107060 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- 172.21.15.132:0/2256220122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0102340 0x7f54f0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- 172.21.15.132:0/2256220122 >> 172.21.15.132:0/2256220122 conn(0x7f54f00fbcb0 msgr2=0x7f54f00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- 172.21.15.132:0/2256220122 shutdown_connections 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- 172.21.15.132:0/2256220122 wait complete. 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 Processor -- start 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- start start 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0104c70 0x7f54f01002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 0x7f54f0100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010d4c0 con 0x7f54f0102340 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010d340 con 0x7f54f0104c70 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f7826640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010d640 con 0x7f54f01075a0 2024-03-18T23:08:38.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f55c2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f55c2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45820/0 (socket says 172.21.15.132:45820) 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.144+0000 7f54f55c2640 1 -- 172.21.15.132:0/3119484935 learned_addr learned my addr 172.21.15.132:0/3119484935 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f4dc1640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0104c70 0x7f54f01002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f5dc3640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 0x7f54f0100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 -- 172.21.15.132:0/3119484935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 msgr2=0x7f54f0100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 0x7f54f0100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 -- 172.21.15.132:0/3119484935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0104c70 msgr2=0x7f54f01002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0104c70 0x7f54f01002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 -- 172.21.15.132:0/3119484935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01010c0 con 0x7f54f0102340 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f55c2640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e9f0 tx=0x7f54ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cd80 con 0x7f54f0102340 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec010910 con 0x7f54f0102340 2024-03-18T23:08:38.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f01a1c80 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec018990 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.145+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a21a0 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.146+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f00fd7a0 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.147+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f54ec010ac0 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.147+0000 7f54e67fc640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 0x7f54dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.148+0000 7f54f4dc1640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 0x7f54dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.148+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f54ec0cf300 con 0x7f54f0102340 2024-03-18T23:08:38.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.148+0000 7f54f4dc1640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 0x7f54dc0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54e0000c00 tx=0x7f54e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.151+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f54ec09bc00 con 0x7f54f0102340 2024-03-18T23:08:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:38 smithi132 ceph-mon[32020]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:38.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/205748866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:38.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3052205277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3823655782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.365+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f0062420 con 0x7f54f0102340 2024-03-18T23:08:38.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.366+0000 7f54e67fc640 1 -- 172.21.15.132:0/3119484935 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7f54ec09fca0 con 0x7f54f0102340 2024-03-18T23:08:38.367 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:38.367 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:08:38.367 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:08:38.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 msgr2=0x7f54dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 0x7f54dc0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f54e0000c00 tx=0x7f54e0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 msgr2=0x7f54f0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e9f0 tx=0x7f54ec00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 shutdown_connections 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f54dc0a2890 0x7f54dc0a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54f01075a0 0x7f54f0100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f0104c70 0x7f54f01002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 --2- 172.21.15.132:0/3119484935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54f0102340 0x7f54f0101cb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 >> 172.21.15.132:0/3119484935 conn(0x7f54f00fbcb0 msgr2=0x7f54f0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 shutdown_connections 2024-03-18T23:08:38.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.368+0000 7f54f7826640 1 -- 172.21.15.132:0/3119484935 wait complete. 2024-03-18T23:08:38.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T23:07:13.853025+0000', 'modified': '2024-03-18T23:07:18.737695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/769602526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 769602526}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 769602526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/559973533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 559973533}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 559973533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6837/3483317240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3483317240}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3483317240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6837/4044803735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 4044803735}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 4044803735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.196:6839/1134284387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1134284387}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1134284387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6839/837888145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 837888145}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 837888145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24493, 24518, 24485]} 2024-03-18T23:08:38.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-18T23:08:38.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T23:08:38.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:38 smithi196 ceph-mon[67089]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:38.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/205748866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3052205277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3823655782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- 172.21.15.132:0/3718187355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c0100730 msgr2=0x7f91c0100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- 172.21.15.132:0/3718187355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c0100730 0x7f91c0100b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009900 tx=0x7f91b002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:38.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- 172.21.15.132:0/3718187355 shutdown_connections 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- 172.21.15.132:0/3718187355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c01022e0 0x7f91c0108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- 172.21.15.132:0/3718187355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0101920 0x7f91c0101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- 172.21.15.132:0/3718187355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c0100730 0x7f91c0100b50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- 172.21.15.132:0/3718187355 >> 172.21.15.132:0/3718187355 conn(0x7f91c00fbcc0 msgr2=0x7f91c00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- 172.21.15.132:0/3718187355 shutdown_connections 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- 172.21.15.132:0/3718187355 wait complete. 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 Processor -- start 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- start start 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0100730 0x7f91c01046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0101920 0x7f91c0102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010c3a0 con 0x7f91c01022e0 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010c220 con 0x7f91c0101920 2024-03-18T23:08:38.485 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.482+0000 7f91c7d90640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91c010c520 con 0x7f91c0100730 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45834/0 (socket says 172.21.15.132:45834) 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 -- 172.21.15.132:0/2864425580 learned_addr learned my addr 172.21.15.132:0/2864425580 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c532b640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0101920 0x7f91c0102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0100730 0x7f91c01046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 -- 172.21.15.132:0/2864425580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0100730 msgr2=0x7f91c01046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0100730 0x7f91c01046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 -- 172.21.15.132:0/2864425580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0101920 msgr2=0x7f91c0102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0101920 0x7f91c0102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 -- 172.21.15.132:0/2864425580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c0103ad0 con 0x7f91c01022e0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c632d640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f91b800eb10 tx=0x7f91b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91b6ffd640 1 -- 172.21.15.132:0/2864425580 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8004b20 con 0x7f91c01022e0 2024-03-18T23:08:38.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91b6ffd640 1 -- 172.21.15.132:0/2864425580 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b8004cc0 con 0x7f91c01022e0 2024-03-18T23:08:38.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c01a1ca0 con 0x7f91c01022e0 2024-03-18T23:08:38.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91b6ffd640 1 -- 172.21.15.132:0/2864425580 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8010760 con 0x7f91c01022e0 2024-03-18T23:08:38.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.483+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c01a21a0 con 0x7f91c01022e0 2024-03-18T23:08:38.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.485+0000 7f91b6ffd640 1 -- 172.21.15.132:0/2864425580 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f91b8005020 con 0x7f91c01022e0 2024-03-18T23:08:38.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 Processor -- start 2024-03-18T23:08:38.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 -- start start 2024-03-18T23:08:38.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca58e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca595fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca4f2de0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca49bc80 con 0x55a7ca58dff0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91ca980740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca4eee80 con 0x55a7ca593b50 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91b6ffd640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 0x7f91ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca58e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45836/0 (socket says 172.21.15.132:45836) 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c5b2c640 1 -- 172.21.15.132:0/3746666457 learned_addr learned my addr 172.21.15.132:0/3746666457 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca595fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c632d640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 0x7f91ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91b6ffd640 1 -- 172.21.15.132:0/2864425580 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f91b80cf1c0 con 0x7f91c01022e0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 -- 172.21.15.132:0/3746666457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 msgr2=0x55a7ca595fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca595fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 -- 172.21.15.132:0/3746666457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 msgr2=0x55a7ca5934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c532b640 1 -- 172.21.15.132:0/3746666457 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ca4adfb0 con 0x55a7ca58dff0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.486+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.487+0000 7f91c632d640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 0x7f91ac0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91b80983e0 tx=0x7f91b806a870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.487+0000 7f91c532b640 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca58e410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91c0104430 tx=0x7f91bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64931da2dab267e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.487+0000 7f91abfff640 1 -- 172.21.15.132:0/3746666457 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc004050 con 0x55a7ca58dff0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.487+0000 7f91abfff640 1 -- 172.21.15.132:0/3746666457 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91bc00fca0 con 0x55a7ca58dff0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.487+0000 7f91abfff640 1 -- 172.21.15.132:0/3746666457 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc003a10 con 0x55a7ca58dff0 2024-03-18T23:08:38.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- 172.21.15.132:0/3746666457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 msgr2=0x55a7ca58e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca58e410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91c0104430 tx=0x7f91bc004730 comp rx=0 tx=0).stop 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- 172.21.15.132:0/3746666457 shutdown_connections 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca595fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca5934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- 172.21.15.132:0/3746666457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca58e410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- 172.21.15.132:0/3746666457 >> 172.21.15.132:0/3746666457 conn(0x55a7ca589560 msgr2=0x55a7ca589960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- 172.21.15.132:0/3746666457 shutdown_connections 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- 172.21.15.132:0/3746666457 wait complete. 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- start start 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 Processor -- start 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca5985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca596b00 con 0x55a7ca58e9e0 2024-03-18T23:08:38.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca4aefc0 con 0x55a7ca58dff0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca596c80 con 0x55a7ca593b50 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91ca980740 10 client.0 _ll_register_callbacks cb 0x55a7ca4eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.488+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45846/0 (socket says 172.21.15.132:45846) 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 -- 172.21.15.132:0/2811321202 learned_addr learned my addr 172.21.15.132:0/2811321202 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c532b640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca5985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c532b640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 -- 172.21.15.132:0/2811321202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 msgr2=0x55a7ca59fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 -- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 msgr2=0x55a7ca5985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca5985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 -- 172.21.15.132:0/2811321202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ca2072d0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009230 tx=0x7f91b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec6df0ca9a18893 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0005d50 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f91b0005220 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91bc00fca0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f91b0005420 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91ca980740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a7ca2d16b0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.489+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91bc00fca0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f91b00038c0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 20 client.15054 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca5a53f0 0x55a7ca5a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 4 client.15054 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 --> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ca500320 con 0x55a7ca5a53f0 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91c532b640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca5a53f0 0x55a7ca5a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91a97fa640 10 client.15054 ms_handle_connect on v2:172.21.15.144:6836/4044803735 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.490+0000 7f91c532b640 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca5a53f0 0x55a7ca5a6860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eae2f9800b4c6212 server_cookie=40b48fb099ba237d in_seq=0 out_seq=0 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91a97fa640 1 -- 172.21.15.132:0/2811321202 <== mds.2 v2:172.21.15.144:6836/4044803735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a7ca500320 con 0x55a7ca5a53f0 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91a97fa640 10 client.15054 handle_command_reply: tid=0 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 client.15054 shutdown 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 msgr2=0x55a7ca598ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009230 tx=0x7f91b0004270 comp rx=0 tx=0).stop 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 shutdown_connections 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca5a53f0 0x55a7ca5a6860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca598ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 --2- 172.21.15.132:0/2811321202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58dff0 0x55a7ca5985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91a97fa640 0 client.15054 ms_handle_reset on v2:172.21.15.144:6836/4044803735 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 >> 172.21.15.132:0/2811321202 conn(0x55a7ca589560 msgr2=0x55a7ca5887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 shutdown_connections 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 1 -- 172.21.15.132:0/2811321202 wait complete. 2024-03-18T23:08:38.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.491+0000 7f91ca980740 20 client.15054 trim_cache size 0 max 16384 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 Processor -- start 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 -- start start 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca58dff0 0x55a7ca5a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 0x55a7ca58ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a4ba0 con 0x55a7ca58e9e0 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a5270 con 0x55a7ca593b50 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91ca980740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a5940 con 0x55a7ca58dff0 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45856/0 (socket says 172.21.15.132:45856) 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 -- 172.21.15.132:0/3999911592 learned_addr learned my addr 172.21.15.132:0/3999911592 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c532b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 0x55a7ca58ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c532b640 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca58dff0 0x55a7ca5a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 -- 172.21.15.132:0/3999911592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca58dff0 msgr2=0x55a7ca5a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca58dff0 0x55a7ca5a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 -- 172.21.15.132:0/3999911592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 msgr2=0x55a7ca58ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 0x55a7ca58ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c5b2c640 1 -- 172.21.15.132:0/3999911592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ca5a0f30 con 0x55a7ca58e9e0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.509+0000 7f91c532b640 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 0x55a7ca58ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f91b002c170 tx=0x7f91b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eb0cf0c5916ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ab7fe640 1 -- 172.21.15.132:0/3999911592 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0000e60 con 0x55a7ca58e9e0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ab7fe640 1 -- 172.21.15.132:0/3999911592 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b0054030 con 0x55a7ca58e9e0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ab7fe640 1 -- 172.21.15.132:0/3999911592 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0005d50 con 0x55a7ca58e9e0 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 -- 172.21.15.132:0/3999911592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 msgr2=0x55a7ca59fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f91b002c170 tx=0x7f91b0000bc0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 -- 172.21.15.132:0/3999911592 shutdown_connections 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca593b50 0x55a7ca58ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59fa70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 --2- 172.21.15.132:0/3999911592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca58dff0 0x55a7ca5a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 -- 172.21.15.132:0/3999911592 >> 172.21.15.132:0/3999911592 conn(0x55a7ca589560 msgr2=0x55a7ca589940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 -- 172.21.15.132:0/3999911592 shutdown_connections 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.510+0000 7f91ca980740 1 -- 172.21.15.132:0/3999911592 wait complete. 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 -- start start 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 Processor -- start 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a66e0 con 0x55a7ca58dff0 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a6010 con 0x55a7ca58e9e0 2024-03-18T23:08:38.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a7ca5a6db0 con 0x55a7ca593b50 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91ca980740 10 client.0 _ll_register_callbacks cb 0x7f91b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45862/0 (socket says 172.21.15.132:45862) 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 -- 172.21.15.132:0/1937700558 learned_addr learned my addr 172.21.15.132:0/1937700558 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 -- 172.21.15.132:0/1937700558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 msgr2=0x55a7ca59ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 -- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58e9e0 msgr2=0x55a7ca59c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.511+0000 7f91c532b640 1 -- 172.21.15.132:0/1937700558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ca59f1c0 con 0x55a7ca58dff0 2024-03-18T23:08:38.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91c532b640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f91bc000c00 tx=0x7f91b002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874c7fe3419076f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00fbe0 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f91bc011050 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b0054030 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f91bc03b070 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a7ca60b670 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b0054030 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f91bc01f850 con 0x55a7ca58dff0 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 20 client.15060 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca60fff0 0x55a7ca60ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 4 client.15060 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:38.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 --> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] -- command(tid 0: {"prefix": "status"}) -- 0x55a7ca4fa670 con 0x55a7ca60fff0 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.512+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca60fff0 0x55a7ca60ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91a97fa640 10 client.15060 ms_handle_connect on v2:172.21.15.144:6836/4044803735 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91c5b2c640 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca60fff0 0x55a7ca60ed00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5fd91cd3ceac3b6 server_cookie=831781966c1d895e in_seq=0 out_seq=0 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91a97fa640 1 -- 172.21.15.132:0/1937700558 <== mds.2 v2:172.21.15.144:6836/4044803735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7ca4fa670 con 0x55a7ca60fff0 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91a97fa640 10 client.15060 handle_command_reply: tid=0 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 client.15060 shutdown 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 msgr2=0x55a7ca59bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f91bc000c00 tx=0x7f91b002cdd0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 shutdown_connections 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x55a7ca60fff0 0x55a7ca60ed00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a7ca593b50 0x55a7ca59ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7ca58e9e0 0x55a7ca59c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 --2- 172.21.15.132:0/1937700558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7ca58dff0 0x55a7ca59bc00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91a97fa640 0 client.15060 ms_handle_reset on v2:172.21.15.144:6836/4044803735 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 >> 172.21.15.132:0/1937700558 conn(0x55a7ca589560 msgr2=0x55a7ca5a4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 shutdown_connections 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 1 -- 172.21.15.132:0/1937700558 wait complete. 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.513+0000 7f91ca980740 20 client.15060 trim_cache size 0 max 16384 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:08:38.521 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 2, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24485, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 80.653159376999994, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 100.430172829 2024-03-18T23:08:38.522 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 msgr2=0x7f91ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 0x7f91ac0a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91b80983e0 tx=0x7f91b806a870 comp rx=0 tx=0).stop 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 msgr2=0x7f91c0103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f91b800eb10 tx=0x7f91b800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 shutdown_connections 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f91ac0a2890 0x7f91ac0a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91c01022e0 0x7f91c0103240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0101920 0x7f91c0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 --2- 172.21.15.132:0/2864425580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91c0100730 0x7f91c01046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 >> 172.21.15.132:0/2864425580 conn(0x7f91c00fbcc0 msgr2=0x7f91c00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 shutdown_connections 2024-03-18T23:08:38.524 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.515+0000 7f91c7d90640 1 -- 172.21.15.132:0/2864425580 wait complete. 2024-03-18T23:08:38.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T23:08:38.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:38 smithi144 ceph-mon[66781]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:38.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/205748866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:08:38.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3052205277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3823655782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 -- 172.21.15.132:0/80795999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 msgr2=0x7f48ec108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 --2- 172.21.15.132:0/80795999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec108b60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009860 tx=0x7f48e402bb70 comp rx=0 tx=0).stop 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 -- 172.21.15.132:0/80795999 shutdown_connections 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 --2- 172.21.15.132:0/80795999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec108b60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 --2- 172.21.15.132:0/80795999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec101910 0x7f48ec101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.628+0000 7f48f3377640 1 --2- 172.21.15.132:0/80795999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec100720 0x7f48ec100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- 172.21.15.132:0/80795999 >> 172.21.15.132:0/80795999 conn(0x7f48ec0fbcb0 msgr2=0x7f48ec0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- 172.21.15.132:0/80795999 shutdown_connections 2024-03-18T23:08:38.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- 172.21.15.132:0/80795999 wait complete. 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 Processor -- start 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- start start 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec101910 0x7f48ec19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10c380 con 0x7f48ec100720 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10c200 con 0x7f48ec1022d0 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f3377640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec10c500 con 0x7f48ec101910 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45880/0 (socket says 172.21.15.132:45880) 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.629+0000 7f48f1113640 1 -- 172.21.15.132:0/548064252 learned_addr learned my addr 172.21.15.132:0/548064252 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f0912640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec101910 0x7f48ec19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1914640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 -- 172.21.15.132:0/548064252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec101910 msgr2=0x7f48ec19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec101910 0x7f48ec19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 -- 172.21.15.132:0/548064252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 msgr2=0x7f48ec1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 -- 172.21.15.132:0/548064252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1a1940 con 0x7f48ec100720 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f1113640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48e800ebc0 tx=0x7f48e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48e27fc640 1 -- 172.21.15.132:0/548064252 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8003b70 con 0x7f48ec100720 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1a1c30 con 0x7f48ec100720 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48e27fc640 1 -- 172.21.15.132:0/548064252 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e8003d10 con 0x7f48ec100720 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48e27fc640 1 -- 172.21.15.132:0/548064252 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8019980 con 0x7f48ec100720 2024-03-18T23:08:38.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.630+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1a2180 con 0x7f48ec100720 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 Processor -- start 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48e27fc640 1 -- 172.21.15.132:0/548064252 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f48e8019b20 con 0x7f48ec100720 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 -- start start 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602036410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x56360203dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563601f9ade0 con 0x5636020369e0 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563601f43c80 con 0x563602035ff0 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563601f96e80 con 0x56360203bb50 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45884/0 (socket says 172.21.15.132:45884) 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f1113640 1 -- 172.21.15.132:0/2386411761 learned_addr learned my addr 172.21.15.132:0/2386411761 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f1914640 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.633+0000 7f48f0912640 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x56360203dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48e27fc640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 0x7f48d80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 -- 172.21.15.132:0/2386411761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 msgr2=0x56360203dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x56360203dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 -- 172.21.15.132:0/2386411761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 msgr2=0x563602036410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602036410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 -- 172.21.15.132:0/2386411761 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563601f55fb0 con 0x5636020369e0 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1914640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 0x7f48d80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48f1113640 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ccd50 tx=0x7f48e8097ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618cb34c326d6446 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48e27fc640 1 -- 172.21.15.132:0/548064252 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f48e80cf450 con 0x7f48ec100720 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48d77fe640 1 -- 172.21.15.132:0/2386411761 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e809ba70 con 0x5636020369e0 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48d77fe640 1 -- 172.21.15.132:0/2386411761 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e80b5a40 con 0x5636020369e0 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.634+0000 7f48d77fe640 1 -- 172.21.15.132:0/2386411761 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8099de0 con 0x5636020369e0 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2386411761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 msgr2=0x56360203b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f1914640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 0x7f48d80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005940 tx=0x7f48e402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ccd50 tx=0x7f48e8097ce0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2386411761 shutdown_connections 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x56360203dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x56360203b4e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2386411761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602036410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2386411761 >> 172.21.15.132:0/2386411761 conn(0x563602031560 msgr2=0x563602031960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2386411761 shutdown_connections 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2386411761 wait complete. 2024-03-18T23:08:38.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 -- start start 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.635+0000 7f48f5bfd740 1 Processor -- start 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602040500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 0x563602040a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56360203eaa0 con 0x56360203bb50 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563601f56fc0 con 0x563602035ff0 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56360203ec20 con 0x5636020369e0 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 10 client.0 _ll_register_callbacks cb 0x563601f92a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f0912640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 0x563602040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f0912640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 0x563602040a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57982/0 (socket says 172.21.15.132:57982) 2024-03-18T23:08:38.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1113640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602040500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45896/0 (socket says 172.21.15.132:45896) 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f0912640 1 -- 172.21.15.132:0/4256199041 learned_addr learned my addr 172.21.15.132:0/4256199041 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 -- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 msgr2=0x563602040a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 0x563602040a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 -- 172.21.15.132:0/4256199041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 msgr2=0x563602040500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602040500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 -- 172.21.15.132:0/4256199041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563601caf2d0 con 0x56360203bb50 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1113640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602040500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f1914640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48e402c0b0 tx=0x7f48e40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb3b96741d442a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e404f020 con 0x56360203bb50 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f48e402c510 con 0x56360203bb50 2024-03-18T23:08:38.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563601fa8320 con 0x56360203bb50 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f48e4039040 con 0x56360203bb50 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 10 client.15072 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.636+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563601d796b0 con 0x56360203bb50 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563601fa8320 con 0x56360203bb50 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f48e403f040 con 0x56360203bb50 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 10 client.15072 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 20 client.15072 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x56360204d2a0 0x56360204e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 4 client.15072 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 --> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563601fa2670 con 0x56360204d2a0 2024-03-18T23:08:38.644 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f1113640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x56360204d2a0 0x56360204e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48d4ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.196:6838/1134284387 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.637+0000 7f48f1113640 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x56360204d2a0 0x56360204e710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c116f6d690f6740e server_cookie=31ff92343bb1d743 in_seq=0 out_seq=0 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48d4ff9640 1 -- 172.21.15.132:0/4256199041 <== mds.1 v2:172.21.15.196:6838/1134284387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563601fa2670 con 0x56360204d2a0 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48d4ff9640 10 client.15072 handle_command_reply: tid=0 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 client.15072 shutdown 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 msgr2=0x563602047a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48e402c0b0 tx=0x7f48e40058b0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 shutdown_connections 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x56360204d2a0 0x56360204e710 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56360203bb50 0x563602047a10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5636020369e0 0x563602040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48d4ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.196:6838/1134284387 2024-03-18T23:08:38.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/4256199041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602040500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 >> 172.21.15.132:0/4256199041 conn(0x563602031560 msgr2=0x5636020307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 shutdown_connections 2024-03-18T23:08:38.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 1 -- 172.21.15.132:0/4256199041 wait complete. 2024-03-18T23:08:38.646 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.638+0000 7f48f5bfd740 20 client.15072 trim_cache size 0 max 16384 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 Processor -- start 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 -- start start 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 0x56360203f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x5636020479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56360204ca50 con 0x563602035ff0 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56360204d120 con 0x5636020369e0 2024-03-18T23:08:38.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56360204d7f0 con 0x56360203bb50 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f1113640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45904/0 (socket says 172.21.15.132:45904) 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f1113640 1 -- 172.21.15.132:0/1526087054 learned_addr learned my addr 172.21.15.132:0/1526087054 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f0912640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 0x56360203f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.656+0000 7f48f1914640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x5636020479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 -- 172.21.15.132:0/1526087054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 msgr2=0x5636020479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x5636020479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 -- 172.21.15.132:0/1526087054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 msgr2=0x56360203f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 0x56360203f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 -- 172.21.15.132:0/1526087054 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563602036dc0 con 0x563602035ff0 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f0912640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 0x56360203f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:08:38.662 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f1113640 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f48e8098590 tx=0x7f48e80cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd942391db0971fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48d6ffd640 1 -- 172.21.15.132:0/1526087054 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e809b600 con 0x563602035ff0 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48d6ffd640 1 -- 172.21.15.132:0/1526087054 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e809c3a0 con 0x563602035ff0 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48d6ffd640 1 -- 172.21.15.132:0/1526087054 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e809b7e0 con 0x563602035ff0 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 -- 172.21.15.132:0/1526087054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 msgr2=0x56360203eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f48e8098590 tx=0x7f48e80cf040 comp rx=0 tx=0).stop 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 -- 172.21.15.132:0/1526087054 shutdown_connections 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x5636020479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636020369e0 0x56360203f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/1526087054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563602035ff0 0x56360203eff0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 -- 172.21.15.132:0/1526087054 >> 172.21.15.132:0/1526087054 conn(0x563602031560 msgr2=0x563602040500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.657+0000 7f48f5bfd740 1 -- 172.21.15.132:0/1526087054 shutdown_connections 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 -- 172.21.15.132:0/1526087054 wait complete. 2024-03-18T23:08:38.663 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 -- start start 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 Processor -- start 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602043070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x563602046420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56360204e590 con 0x5636020369e0 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56360204dec0 con 0x563602035ff0 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56360204ec60 con 0x56360203bb50 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f5bfd740 10 client.0 _ll_register_callbacks cb 0x7f48e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45912/0 (socket says 172.21.15.132:45912) 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 -- 172.21.15.132:0/2818507617 learned_addr learned my addr 172.21.15.132:0/2818507617 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f1914640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x563602046420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f1113640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602043070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 -- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 msgr2=0x563602046420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x563602046420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.658+0000 7f48f0912640 1 -- 172.21.15.132:0/2818507617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 msgr2=0x563602043070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f0912640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602043070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f0912640 1 -- 172.21.15.132:0/2818507617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563602046a10 con 0x5636020369e0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f0912640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x563602041970 tx=0x7f48dc010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca10740f54289917 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc004020 con 0x5636020369e0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f48dc0041c0 con 0x5636020369e0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e809c3a0 con 0x5636020369e0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f48dc0043c0 con 0x5636020369e0 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 10 client.15078 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563602047e10 con 0x5636020369e0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e809c3a0 con 0x5636020369e0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f48dc0047a0 con 0x5636020369e0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 10 client.15078 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 20 client.15078 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5636020b7af0 0x5636020b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 4 client.15078 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.659+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 --> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] -- command(tid 0: {"prefix": "status"}) -- 0x563601f20a00 con 0x5636020b7af0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f1113640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5636020b7af0 0x5636020b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f1113640 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5636020b7af0 0x5636020b6800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4baa308bf6488c5 server_cookie=51e4cde64181d2bb in_seq=0 out_seq=0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48d4ff9640 10 client.15078 ms_handle_connect on v2:172.21.15.196:6838/1134284387 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48d4ff9640 1 -- 172.21.15.132:0/2818507617 <== mds.1 v2:172.21.15.196:6838/1134284387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563601f20a00 con 0x5636020b7af0 2024-03-18T23:08:38.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48d4ff9640 10 client.15078 handle_command_reply: tid=0 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 client.15078 shutdown 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 msgr2=0x5636020435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x563602041970 tx=0x7f48dc010b40 comp rx=0 tx=0).stop 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 shutdown_connections 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56360203bb50 0x563602046420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5636020b7af0 0x5636020b6800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636020369e0 0x5636020435b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 --2- 172.21.15.132:0/2818507617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563602035ff0 0x563602043070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48d4ff9640 0 client.15078 ms_handle_reset on v2:172.21.15.196:6838/1134284387 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 >> 172.21.15.132:0/2818507617 conn(0x563602031560 msgr2=0x563602037f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 shutdown_connections 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 1 -- 172.21.15.132:0/2818507617 wait complete. 2024-03-18T23:08:38.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.660+0000 7f48f5bfd740 20 client.15078 trim_cache size 0 max 16384 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 1, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24493, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 81.804521780000002, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:08:38.668 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 97.783101701000007 2024-03-18T23:08:38.669 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:08:38.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 msgr2=0x7f48d80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 0x7f48d80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005940 tx=0x7f48e402e040 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 msgr2=0x7f48ec199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48e800ebc0 tx=0x7f48e800c6a0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 shutdown_connections 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48d80a2890 0x7f48d80a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48ec1022d0 0x7f48ec1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48ec101910 0x7f48ec19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 --2- 172.21.15.132:0/548064252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48ec100720 0x7f48ec199d30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 >> 172.21.15.132:0/548064252 conn(0x7f48ec0fbcb0 msgr2=0x7f48ec0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 shutdown_connections 2024-03-18T23:08:38.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.661+0000 7f48f3377640 1 -- 172.21.15.132:0/548064252 wait complete. 2024-03-18T23:08:38.671 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T23:08:38.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 -- 172.21.15.132:0/1938817140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4314101910 msgr2=0x7f4314101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 --2- 172.21.15.132:0/1938817140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4314101910 0x7f4314101d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4308009860 tx=0x7f430802bb70 comp rx=0 tx=0).stop 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 -- 172.21.15.132:0/1938817140 shutdown_connections 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 --2- 172.21.15.132:0/1938817140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141022d0 0x7f4314108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 --2- 172.21.15.132:0/1938817140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4314101910 0x7f4314101d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 --2- 172.21.15.132:0/1938817140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314100720 0x7f4314100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 -- 172.21.15.132:0/1938817140 >> 172.21.15.132:0/1938817140 conn(0x7f43140fbcb0 msgr2=0x7f43140fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 -- 172.21.15.132:0/1938817140 shutdown_connections 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.774+0000 7f4319337640 1 -- 172.21.15.132:0/1938817140 wait complete. 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 Processor -- start 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 -- start start 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314101910 0x7f431419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141022d0 0x7f431419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c380 con 0x7f4314100720 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c200 con 0x7f4314101910 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4319337640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f431410c500 con 0x7f43141022d0 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45936/0 (socket says 172.21.15.132:45936) 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f4312ffd640 1 -- 172.21.15.132:0/934787396 learned_addr learned my addr 172.21.15.132:0/934787396 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f43127fc640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314101910 0x7f431419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.775+0000 7f43137fe640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141022d0 0x7f431419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 -- 172.21.15.132:0/934787396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141022d0 msgr2=0x7f431419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141022d0 0x7f431419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 -- 172.21.15.132:0/934787396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314101910 msgr2=0x7f431419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314101910 0x7f431419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 -- 172.21.15.132:0/934787396 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431419f7d0 con 0x7f4314100720 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4312ffd640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f430400ebc0 tx=0x7f430400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.778 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f42fbfff640 1 -- 172.21.15.132:0/934787396 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304003b70 con 0x7f4314100720 2024-03-18T23:08:38.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431419fac0 con 0x7f4314100720 2024-03-18T23:08:38.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f42fbfff640 1 -- 172.21.15.132:0/934787396 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4304003d10 con 0x7f4314100720 2024-03-18T23:08:38.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f42fbfff640 1 -- 172.21.15.132:0/934787396 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304019980 con 0x7f4314100720 2024-03-18T23:08:38.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.776+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a0030 con 0x7f4314100720 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f42fbfff640 1 -- 172.21.15.132:0/934787396 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f4304019b20 con 0x7f4314100720 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 Processor -- start 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 -- start start 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5596db608de0 con 0x5596db6a3ff0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596db5b1c80 con 0x5596db6a9b50 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f431bffb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596db604e80 con 0x5596db6a49e0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.778+0000 7f42fbfff640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 0x7f42fc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45938/0 (socket says 172.21.15.132:45938) 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 -- 172.21.15.132:0/1481250789 learned_addr learned my addr 172.21.15.132:0/1481250789 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 0x7f42fc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43127fc640 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f42fbfff640 1 -- 172.21.15.132:0/934787396 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f43040cf390 con 0x7f4314100720 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 -- 172.21.15.132:0/1481250789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 msgr2=0x5596db6a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 -- 172.21.15.132:0/1481250789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 msgr2=0x5596db6abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 -- 172.21.15.132:0/1481250789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596db5c3fb0 con 0x5596db6a3ff0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f430c011aa0 tx=0x7f430c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb07a3eb7956b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f43137fe640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 0x7f42fc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f430c011fd0 tx=0x7f430c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f42f97fa640 1 -- 172.21.15.132:0/1481250789 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c011050 con 0x5596db6a3ff0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f42f97fa640 1 -- 172.21.15.132:0/1481250789 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c009a80 con 0x5596db6a3ff0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.779+0000 7f42f97fa640 1 -- 172.21.15.132:0/1481250789 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c01dd80 con 0x5596db6a3ff0 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- 172.21.15.132:0/1481250789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 msgr2=0x5596db6a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f430c011aa0 tx=0x7f430c011f60 comp rx=0 tx=0).stop 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- 172.21.15.132:0/1481250789 shutdown_connections 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 --2- 172.21.15.132:0/1481250789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6a4410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- 172.21.15.132:0/1481250789 >> 172.21.15.132:0/1481250789 conn(0x5596db69f560 msgr2=0x5596db69f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- 172.21.15.132:0/1481250789 shutdown_connections 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- 172.21.15.132:0/1481250789 wait complete. 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 -- start start 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.780+0000 7f431bffb740 1 Processor -- start 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5596db6acaa0 con 0x5596db6a3ff0 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596db5c4fc0 con 0x5596db6a49e0 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596db6acc20 con 0x5596db6a9b50 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 10 client.0 _ll_register_callbacks cb 0x5596db600a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45944/0 (socket says 172.21.15.132:45944) 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 -- 172.21.15.132:0/1195283939 learned_addr learned my addr 172.21.15.132:0/1195283939 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f43127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f43137fe640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 -- 172.21.15.132:0/1195283939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 msgr2=0x5596db6b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 -- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 msgr2=0x5596db6ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 -- 172.21.15.132:0/1195283939 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596db31d2d0 con 0x5596db6a3ff0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43040985b0 tx=0x7f4304004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bab9809ee8c6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f42f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040b3030 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f43040071c0 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c009a80 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4304086040 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 10 client.15090 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.781+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596db3e76b0 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430c009a80 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f4304096290 con 0x5596db6a3ff0 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 10 client.15090 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 20 client.15090 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db6bb1e0 0x5596db6bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 4 client.15090 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:08:38.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596db616320 con 0x5596db6bb1e0 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f43127fc640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db6bb1e0 0x5596db6bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f42f6ff5640 10 client.15090 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.782+0000 7f43127fc640 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db6bb1e0 0x5596db6bc650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e7a60900edf53d server_cookie=970cd532bad1203a in_seq=0 out_seq=0 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1195283939 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5596db616320 con 0x5596db6bb1e0 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f42f6ff5640 10 client.15090 handle_command_reply: tid=0 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 client.15090 shutdown 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 msgr2=0x5596db6ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43040985b0 tx=0x7f4304004910 comp rx=0 tx=0).stop 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 shutdown_connections 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db6bb1e0 0x5596db6bc650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 --2- 172.21.15.132:0/1195283939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ae470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f42f6ff5640 0 client.15090 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:08:38.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 >> 172.21.15.132:0/1195283939 conn(0x5596db69f560 msgr2=0x5596db69e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 shutdown_connections 2024-03-18T23:08:38.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 1 -- 172.21.15.132:0/1195283939 wait complete. 2024-03-18T23:08:38.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.783+0000 7f431bffb740 20 client.15090 trim_cache size 0 max 16384 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 Processor -- start 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 -- start start 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5596db6ba990 con 0x5596db6a3ff0 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596db6bb060 con 0x5596db6a49e0 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f431bffb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596db6bb730 con 0x5596db6a9b50 2024-03-18T23:08:38.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45954/0 (socket says 172.21.15.132:45954) 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.801+0000 7f4312ffd640 1 -- 172.21.15.132:0/90614239 learned_addr learned my addr 172.21.15.132:0/90614239 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f43127fc640 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f43137fe640 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 -- 172.21.15.132:0/90614239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 msgr2=0x5596db6a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 -- 172.21.15.132:0/90614239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 msgr2=0x5596db6b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 -- 172.21.15.132:0/90614239 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596db6af460 con 0x5596db6a3ff0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f4312ffd640 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4304098040 tx=0x7f4304000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf44210466d0ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f42f8ff9640 1 -- 172.21.15.132:0/90614239 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040b3070 con 0x5596db6a3ff0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f42f8ff9640 1 -- 172.21.15.132:0/90614239 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43040b9070 con 0x5596db6a3ff0 2024-03-18T23:08:38.807 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f42f8ff9640 1 -- 172.21.15.132:0/90614239 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040ba270 con 0x5596db6a3ff0 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f431bffb740 1 -- 172.21.15.132:0/90614239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 msgr2=0x5596db6ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f431bffb740 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4304098040 tx=0x7f4304000f40 comp rx=0 tx=0).stop 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.802+0000 7f431bffb740 1 -- 172.21.15.132:0/90614239 shutdown_connections 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a9b50 0x5596db6a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a49e0 0x5596db6b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- 172.21.15.132:0/90614239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6ada90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- 172.21.15.132:0/90614239 >> 172.21.15.132:0/90614239 conn(0x5596db69f560 msgr2=0x5596db69f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- 172.21.15.132:0/90614239 shutdown_connections 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- 172.21.15.132:0/90614239 wait complete. 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- start start 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 Processor -- start 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.808 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5596db6bc4d0 con 0x5596db6a3ff0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596db6bbe00 con 0x5596db6a9b50 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596db6bcba0 con 0x5596db6a49e0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f431bffb740 10 client.0 _ll_register_callbacks cb 0x7f4304003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45966/0 (socket says 172.21.15.132:45966) 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 -- 172.21.15.132:0/1597388755 learned_addr learned my addr 172.21.15.132:0/1597388755 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f43137fe640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f43127fc640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 -- 172.21.15.132:0/1597388755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 msgr2=0x5596db6b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 -- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 msgr2=0x5596db6b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:08:38.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.803+0000 7f4312ffd640 1 -- 172.21.15.132:0/1597388755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596db6b50d0 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f4312ffd640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f43040ccdd0 tx=0x7f43040096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b51c6babe57cfa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430407f020 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4304004d60 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43040b9070 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43040b3070 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 10 client.15096 fetch_fsmap learned FSMap version 20 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5596db7213c0 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43040b9070 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f430409b1c0 con 0x5596db6a3ff0 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 10 client.15096 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:08:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 20 client.15096 populate_metadata read hostname 'smithi132' 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db7259e0 0x5596db7246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 4 client.15096 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "status"}) -- 0x5596db610670 con 0x5596db7259e0 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.804+0000 7f43127fc640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db7259e0 0x5596db7246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f42f6ff5640 10 client.15096 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f43127fc640 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db7259e0 0x5596db7246f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40de9fd02c22eb43 server_cookie=7539396b2aee1a6a in_seq=0 out_seq=0 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f42f6ff5640 1 -- 172.21.15.132:0/1597388755 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5596db610670 con 0x5596db7259e0 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f42f6ff5640 10 client.15096 handle_command_reply: tid=0 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 client.15096 shutdown 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 msgr2=0x5596db6b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f43040ccdd0 tx=0x7f43040096c0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 shutdown_connections 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5596db7259e0 0x5596db7246f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596db6a9b50 0x5596db6b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596db6a49e0 0x5596db6b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f42f6ff5640 0 client.15096 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 --2- 172.21.15.132:0/1597388755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596db6a3ff0 0x5596db6b1580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 >> 172.21.15.132:0/1597388755 conn(0x5596db69f560 msgr2=0x5596db6a5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 shutdown_connections 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 1 -- 172.21.15.132:0/1597388755 wait complete. 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f431bffb740 20 client.15096 trim_cache size 0 max 16384 2024-03-18T23:08:38.812 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 0, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24518, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 83.968461700000006, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 89.709967601000002 2024-03-18T23:08:38.813 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:08:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 msgr2=0x7f42fc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 0x7f42fc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f430c011fd0 tx=0x7f430c01e000 comp rx=0 tx=0).stop 2024-03-18T23:08:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 msgr2=0x7f4314069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.805+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f430400ebc0 tx=0x7f430400c6a0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 shutdown_connections 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f42fc0a2890 0x7f42fc0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43141022d0 0x7f431419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4314101910 0x7f431419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 --2- 172.21.15.132:0/934787396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100720 0x7f4314069510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 >> 172.21.15.132:0/934787396 conn(0x7f43140fbcb0 msgr2=0x7f4314104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 shutdown_connections 2024-03-18T23:08:38.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:38.806+0000 7f4319337640 1 -- 172.21.15.132:0/934787396 wait complete. 2024-03-18T23:08:38.816 INFO:tasks.cephadm:Setup complete, yielding 2024-03-18T23:08:38.816 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T23:08:38.826 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:08:38.826 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-18T23:08:39.064 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:39.392 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:39 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3119484935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:39.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:39 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3119484935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:39 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3119484935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:08:40.359 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:40 smithi132 ceph-mon[32020]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:40.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:40 smithi196 ceph-mon[67089]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:40 smithi144 ceph-mon[66781]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 -- 172.21.15.132:0/3660328289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 msgr2=0x7f675c102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 --2- 172.21.15.132:0/3660328289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c102910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f674c0099c0 tx=0x7f674c009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 -- 172.21.15.132:0/3660328289 shutdown_connections 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 --2- 172.21.15.132:0/3660328289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c1012b0 0x7f675c1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 --2- 172.21.15.132:0/3660328289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c102e50 0x7f675c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 --2- 172.21.15.132:0/3660328289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c102910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 -- 172.21.15.132:0/3660328289 >> 172.21.15.132:0/3660328289 conn(0x7f675c0fc870 msgr2=0x7f675c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:40.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 -- 172.21.15.132:0/3660328289 shutdown_connections 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.608+0000 7f67624c6700 1 -- 172.21.15.132:0/3660328289 wait complete. 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 Processor -- start 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 -- start start 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c102e50 0x7f675c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10ce20 con 0x7f675c1024b0 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10cca0 con 0x7f675c102e50 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f67624c6700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10cfa0 con 0x7f675c1012b0 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f675bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f675b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45984/0 (socket says 172.21.15.132:45984) 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f675bfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:58070/0 (socket says 172.21.15.132:58070) 2024-03-18T23:08:40.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.609+0000 7f675b7fe700 1 -- 172.21.15.132:0/128321911 learned_addr learned my addr 172.21.15.132:0/128321911 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f6760a63700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c102e50 0x7f675c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 -- 172.21.15.132:0/128321911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c102e50 msgr2=0x7f675c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c102e50 0x7f675c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 -- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 msgr2=0x7f675c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 -- 172.21.15.132:0/128321911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c06ab10 con 0x7f675c1012b0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675b7fe700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f675bfff700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9c0 tx=0x7f675000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750004220 con 0x7f675c1012b0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67500043c0 con 0x7f675c1012b0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c06ae00 con 0x7f675c1012b0 2024-03-18T23:08:40.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750003d10 con 0x7f675c1012b0 2024-03-18T23:08:40.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.610+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c06b390 con 0x7f675c1012b0 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.611+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c068fd0 con 0x7f675c1012b0 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.612+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f675000c7c0 con 0x7f675c1012b0 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.613+0000 7f67597fa700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 0x7f67440a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.613+0000 7f675b7fe700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 0x7f67440a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.613+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f67500ced80 con 0x7f675c1012b0 2024-03-18T23:08:40.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.613+0000 7f675b7fe700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 0x7f67440a4e60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f674c0036c0 tx=0x7f674c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:40.617 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.617+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f675009bcd0 con 0x7f675c1012b0 2024-03-18T23:08:40.829 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.829+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f675c02dcc0 con 0x7f67440a29a0 2024-03-18T23:08:40.830 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.830+0000 7f67597fa700 1 -- 172.21.15.132:0/128321911 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f675c02dcc0 con 0x7f67440a29a0 2024-03-18T23:08:40.830 INFO:teuthology.orchestra.run.smithi132.stdout:Backend: cephadm 2024-03-18T23:08:40.831 INFO:teuthology.orchestra.run.smithi132.stdout:Available: Yes 2024-03-18T23:08:40.831 INFO:teuthology.orchestra.run.smithi132.stdout:Paused: No 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 msgr2=0x7f67440a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 0x7f67440a4e60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f674c0036c0 tx=0x7f674c005fb0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 msgr2=0x7f675c197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f675000e9c0 tx=0x7f675000ecd0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 shutdown_connections 2024-03-18T23:08:40.833 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f67440a29a0 0x7f67440a4e60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f675c102e50 0x7f675c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f675c1012b0 0x7f675c197430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 --2- 172.21.15.132:0/128321911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f675c1024b0 0x7f675c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 >> 172.21.15.132:0/128321911 conn(0x7f675c0fc870 msgr2=0x7f675c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 shutdown_connections 2024-03-18T23:08:40.834 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:40.832+0000 7f67624c6700 1 -- 172.21.15.132:0/128321911 wait complete. 2024-03-18T23:08:41.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-18T23:08:41.762 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:42.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:42 smithi196 ceph-mon[67089]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:42.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:42 smithi196 ceph-mon[67089]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:42.569 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:42 smithi132 ceph-mon[32020]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:42.569 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:42 smithi132 ceph-mon[32020]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:42 smithi144 ceph-mon[66781]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:42.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:42 smithi144 ceph-mon[66781]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 -- 172.21.15.132:0/2523821976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 msgr2=0x7f0f58101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2523821976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58101660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0f480099c0 tx=0x7f0f48009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 -- 172.21.15.132:0/2523821976 shutdown_connections 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2523821976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102de0 0x7f0f58109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2523821976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58101660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2523821976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102440 0x7f0f581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 -- 172.21.15.132:0/2523821976 >> 172.21.15.132:0/2523821976 conn(0x7f0f580fc800 msgr2=0x7f0f580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 -- 172.21.15.132:0/2523821976 shutdown_connections 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.357+0000 7f0f5d217700 1 -- 172.21.15.132:0/2523821976 wait complete. 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.358+0000 7f0f5d217700 1 Processor -- start 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.358+0000 7f0f5d217700 1 -- start start 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.358+0000 7f0f5d217700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.358+0000 7f0f5d217700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 0x7f0f58103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.358+0000 7f0f5d217700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102de0 0x7f0f58103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5d217700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5810ce00 con 0x7f0f58101240 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5d217700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5810cc80 con 0x7f0f58102440 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5d217700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5810cf80 con 0x7f0f58102de0 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60504/0 (socket says 172.21.15.132:60504) 2024-03-18T23:08:43.361 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 -- 172.21.15.132:0/2204879184 learned_addr learned my addr 172.21.15.132:0/2204879184 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5659c700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 0x7f0f58103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5759e700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102de0 0x7f0f58103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 -- 172.21.15.132:0/2204879184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102de0 msgr2=0x7f0f58103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102de0 0x7f0f58103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 -- 172.21.15.132:0/2204879184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 msgr2=0x7f0f58103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 0x7f0f58103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 -- 172.21.15.132:0/2204879184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f581045a0 con 0x7f0f58101240 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5659c700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 0x7f0f58103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f56d9d700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f48003790 tx=0x7f0f48009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48009e60 con 0x7f0f58101240 2024-03-18T23:08:43.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48003960 con 0x7f0f58101240 2024-03-18T23:08:43.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f581a2680 con 0x7f0f58101240 2024-03-18T23:08:43.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480055e0 con 0x7f0f58101240 2024-03-18T23:08:43.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.359+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a2b60 con 0x7f0f58101240 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.360+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58068fd0 con 0x7f0f58101240 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.361+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f0f48003b10 con 0x7f0f58101240 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.362+0000 7f0f3ffff700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 0x7f0f400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.362+0000 7f0f5659c700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 0x7f0f400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.362+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0f480cfaa0 con 0x7f0f58101240 2024-03-18T23:08:43.364 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.362+0000 7f0f5659c700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 0x7f0f400a4de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c011490 tx=0x7f0f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:43.366 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.366+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f48013070 con 0x7f0f58101240 2024-03-18T23:08:43.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.571+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0f580fe9c0 con 0x7f0f400a2920 2024-03-18T23:08:43.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.585+0000 7f0f3ffff700 1 -- 172.21.15.132:0/2204879184 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f0f580fe9c0 con 0x7f0f400a2920 2024-03-18T23:08:43.585 INFO:teuthology.orchestra.run.smithi132.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.a smithi132 running (97s) 15s ago 97s 15.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e1cfe9ae8e6f 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.b smithi144 running (94s) 28s ago 94s 25.6M - 19.0.0-2206-gb6ae2491 2312c8e42d00 4215ab18d2ad 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.c smithi196 running (108s) 28s ago 108s 18.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 0131005cfb2f 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.d smithi132 running (116s) 15s ago 116s 16.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 30ccf165fb8d 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.e smithi144 running (114s) 28s ago 113s 16.0M - 19.0.0-2206-gb6ae2491 2312c8e42d00 460f266c3d51 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.f smithi196 running (111s) 28s ago 111s 19.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 8977edd44b39 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.g smithi132 running (100s) 15s ago 99s 15.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e81fcc0c83d3 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.h smithi144 running (105s) 28s ago 105s 17.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 19b963446709 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.i smithi196 running (103s) 28s ago 102s 17.7M - 19.0.0-2206-gb6ae2491 2312c8e42d00 963dfdec843c 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.j smithi132 running (119s) 15s ago 119s 16.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 c4ee62bb361c 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.k smithi144 running (2m) 28s ago 2m 15.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 7cc6ceffd3ac 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mds.l smithi196 running (2m) 28s ago 2m 13.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 12f87cf7218f 2024-03-18T23:08:43.586 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.x smithi132 *:9283,8765 running (9m) 15s ago 9m 513M - 19.0.0-2206-gb6ae2491 2312c8e42d00 a033519837ae 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.y smithi144 *:8443,8765 running (7m) 28s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 9825a78361bf 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:mgr.z smithi196 *:8443,8765 running (7m) 28s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 6268e0ea3f0e 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:mon.a smithi132 running (9m) 15s ago 9m 48.3M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 b748b36d2681 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:mon.b smithi144 running (7m) 28s ago 7m 41.6M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 b5996aff448e 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:mon.c smithi196 running (7m) 28s ago 7m 38.8M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 54ceaf63f9e5 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.0 smithi132 running (6m) 15s ago 6m 48.5M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 a5510b1f5087 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.1 smithi144 running (6m) 28s ago 6m 56.6M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 5f02ad44985e 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.2 smithi196 running (6m) 28s ago 6m 50.4M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 53c78363265b 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.3 smithi132 running (5m) 15s ago 5m 49.2M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 8533e7dacd31 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.4 smithi144 running (5m) 28s ago 5m 45.6M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 5078d3deb68a 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.5 smithi196 running (4m) 28s ago 4m 55.1M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 1e700265a99f 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.6 smithi132 running (4m) 15s ago 4m 47.6M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 0a97582946a6 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.7 smithi144 running (4m) 28s ago 4m 54.7M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 8143c71c6ebb 2024-03-18T23:08:43.587 INFO:teuthology.orchestra.run.smithi132.stdout:osd.8 smithi196 running (3m) 28s ago 3m 46.5M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 05fa8d9ec3cf 2024-03-18T23:08:43.588 INFO:teuthology.orchestra.run.smithi132.stdout:osd.9 smithi132 running (3m) 15s ago 3m 54.7M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 57e5a5824051 2024-03-18T23:08:43.588 INFO:teuthology.orchestra.run.smithi132.stdout:osd.10 smithi144 running (2m) 28s ago 2m 59.4M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 266983be1ae6 2024-03-18T23:08:43.588 INFO:teuthology.orchestra.run.smithi132.stdout:osd.11 smithi196 running (2m) 28s ago 2m 50.2M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 cf88be764c4b 2024-03-18T23:08:43.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 msgr2=0x7f0f400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 0x7f0f400a4de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c011490 tx=0x7f0f4c009260 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 msgr2=0x7f0f58105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0f48003790 tx=0x7f0f48009580 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 shutdown_connections 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0f400a2920 0x7f0f400a4de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f58102de0 0x7f0f58103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f58101240 0x7f0f58105160 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 --2- 172.21.15.132:0/2204879184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f58102440 0x7f0f58103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 >> 172.21.15.132:0/2204879184 conn(0x7f0f580fc800 msgr2=0x7f0f580fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 shutdown_connections 2024-03-18T23:08:43.589 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:43.587+0000 7f0f5d217700 1 -- 172.21.15.132:0/2204879184 wait complete. 2024-03-18T23:08:44.193 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:44 smithi132 ceph-mon[32020]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:44.221 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-18T23:08:44.462 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:44.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:44 smithi196 ceph-mon[67089]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:44 smithi144 ceph-mon[66781]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-18T23:08:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:45 smithi196 ceph-mon[67089]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:45.581 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:45 smithi132 ceph-mon[32020]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:45.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:45 smithi144 ceph-mon[66781]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/3676579356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0103540 msgr2=0x7f3fc01039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/3676579356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0103540 0x7f3fc01039a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099c0 tx=0x7f3fbc009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/3676579356 shutdown_connections 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/3676579356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0102340 0x7f3fc0102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/3676579356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/3676579356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0103540 0x7f3fc01039a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.145+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/3676579356 >> 172.21.15.132:0/3676579356 conn(0x7f3fc00fd920 msgr2=0x7f3fc00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/3676579356 shutdown_connections 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/3676579356 wait complete. 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 Processor -- start 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- start start 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 0x7f3fc0074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc0074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc010df30 con 0x7f3fc0102340 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc010ddb0 con 0x7f3fc0103540 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc7f0a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc010e0b0 con 0x7f3fc0103ee0 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc5ca6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc54a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 0x7f3fc0074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.146+0000 7f3fc5ca6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60524/0 (socket says 172.21.15.132:60524) 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc5ca6700 1 -- 172.21.15.132:0/713728725 learned_addr learned my addr 172.21.15.132:0/713728725 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc64a7700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc0074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 -- 172.21.15.132:0/713728725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 msgr2=0x7f3fc0074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc0074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 -- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 msgr2=0x7f3fc00760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 -- 172.21.15.132:0/713728725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc00754f0 con 0x7f3fc0103540 2024-03-18T23:08:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc64a7700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc0074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc5ca6700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.147+0000 7f3fc54a5700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 0x7f3fc0074700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc003790 tx=0x7f3fbc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.152+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc009e60 con 0x7f3fc0103540 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.152+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fbc003960 con 0x7f3fc0103540 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.152+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc01a26a0 con 0x7f3fc0103540 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.152+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc005560 con 0x7f3fc0103540 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.152+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01a2b80 con 0x7f3fc0103540 2024-03-18T23:08:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.153+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc002dbb0 con 0x7f3fc0103540 2024-03-18T23:08:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.154+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3fbc00faa0 con 0x7f3fc0103540 2024-03-18T23:08:46.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.154+0000 7f3fb6ffd700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 0x7f3fac0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:46.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.154+0000 7f3fc5ca6700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 0x7f3fac0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:46.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.155+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f3fbc0cf460 con 0x7f3fc0103540 2024-03-18T23:08:46.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.155+0000 7f3fc5ca6700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 0x7f3fac0a4e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005950 tx=0x7f3fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:46.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.159+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3fbc013070 con 0x7f3fc0103540 2024-03-18T23:08:46.160 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:46 smithi132 ceph-mon[32020]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:46.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.368+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3fc0062750 con 0x7f3fac0a29a0 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.374+0000 7f3fb6ffd700 1 -- 172.21.15.132:0/713728725 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3fc0062750 con 0x7f3fac0a29a0 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stdout:mds.all 12/12 31s ago 2m smithi132=a;smithi132=d;smithi132=g;smithi132=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stdout:mgr 3/3 31s ago 7m smithi132=x;smithi144=y;smithi196=z;count:3 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stdout:mon 3/3 31s ago 7m smithi132:172.21.15.132=a;smithi144:172.21.15.144=b;smithi196:172.21.15.196=c;count:3 2024-03-18T23:08:46.375 INFO:teuthology.orchestra.run.smithi132.stdout:osd 12 31s ago - 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 msgr2=0x7f3fac0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 0x7f3fac0a4e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005950 tx=0x7f3fb0009260 comp rx=0 tx=0).stop 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 msgr2=0x7f3fc0074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 0x7f3fc0074700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc003790 tx=0x7f3fbc009580 comp rx=0 tx=0).stop 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 shutdown_connections 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f3fac0a29a0 0x7f3fac0a4e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fc0103ee0 0x7f3fc0074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0102340 0x7f3fc00760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 --2- 172.21.15.132:0/713728725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc0103540 0x7f3fc0074700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:46.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 >> 172.21.15.132:0/713728725 conn(0x7f3fc00fd920 msgr2=0x7f3fc0106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:46.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 shutdown_connections 2024-03-18T23:08:46.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:46.376+0000 7f3fc7f0a700 1 -- 172.21.15.132:0/713728725 wait complete. 2024-03-18T23:08:46.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:46 smithi196 ceph-mon[67089]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:46 smithi144 ceph-mon[66781]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:08:47.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-18T23:08:47.280 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:48.323 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:48 smithi132 ceph-mon[32020]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:48.323 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:48 smithi132 ceph-mon[32020]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:48.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:48 smithi196 ceph-mon[67089]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:48.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:48 smithi196 ceph-mon[67089]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:48.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:48 smithi144 ceph-mon[66781]: from='client.24971 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:48.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:48 smithi144 ceph-mon[66781]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 -- 172.21.15.132:0/353548973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124102dd0 msgr2=0x7f7124109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 --2- 172.21.15.132:0/353548973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124102dd0 0x7f7124109680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f711c00b0d0 tx=0x7f711c00b3e0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 -- 172.21.15.132:0/353548973 shutdown_connections 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 --2- 172.21.15.132:0/353548973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124102dd0 0x7f7124109680 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 --2- 172.21.15.132:0/353548973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124101230 0x7f7124101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 --2- 172.21.15.132:0/353548973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 -- 172.21.15.132:0/353548973 >> 172.21.15.132:0/353548973 conn(0x7f71240fc7d0 msgr2=0x7f71240fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 -- 172.21.15.132:0/353548973 shutdown_connections 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.834+0000 7f71290f1700 1 -- 172.21.15.132:0/353548973 wait complete. 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 Processor -- start 2024-03-18T23:08:48.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 -- start start 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124102dd0 0x7f712419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f712410cdf0 con 0x7f7124101230 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f712410cc70 con 0x7f7124102430 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f71290f1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f712410cf70 con 0x7f7124102dd0 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f7122d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60550/0 (socket says 172.21.15.132:60550) 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f7122d9d700 1 -- 172.21.15.132:0/2457102393 learned_addr learned my addr 172.21.15.132:0/2457102393 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f712359e700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124102dd0 0x7f712419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 -- 172.21.15.132:0/2457102393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124102dd0 msgr2=0x7f712419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:48.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124102dd0 0x7f712419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 -- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 msgr2=0x7f71241984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f711a59c700 1 -- 172.21.15.132:0/2457102393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71241a0120 con 0x7f7124102430 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.835+0000 7f7122d9d700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f711a59c700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124198a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f710c00e9f0 tx=0x7f710c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00c9c0 con 0x7f7124102430 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241a0410 con 0x7f7124102430 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f710c00cb60 con 0x7f7124102430 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c018680 con 0x7f7124102430 2024-03-18T23:08:48.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.836+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a0950 con 0x7f7124102430 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.837+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7124068fd0 con 0x7f7124102430 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.838+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f710c047a90 con 0x7f7124102430 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.838+0000 7f7120d99700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 0x7f71100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.838+0000 7f7122d9d700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 0x7f71100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.839+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f710c0cecc0 con 0x7f7124102430 2024-03-18T23:08:48.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.839+0000 7f7122d9d700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 0x7f71100a4de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7114005950 tx=0x7f7114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:48.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:48.842+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f710c09bbb0 con 0x7f7124102430 2024-03-18T23:08:49.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.050+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f712406af60 con 0x7f71100a2920 2024-03-18T23:08:49.060 INFO:teuthology.orchestra.run.smithi132.stdout:HOST ADDR LABELS STATUS 2024-03-18T23:08:49.060 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132 172.21.15.132 2024-03-18T23:08:49.060 INFO:teuthology.orchestra.run.smithi132.stdout:smithi144 172.21.15.144 2024-03-18T23:08:49.061 INFO:teuthology.orchestra.run.smithi132.stdout:smithi196 172.21.15.196 2024-03-18T23:08:49.061 INFO:teuthology.orchestra.run.smithi132.stdout:3 hosts in cluster 2024-03-18T23:08:49.061 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.060+0000 7f7120d99700 1 -- 172.21.15.132:0/2457102393 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f712406af60 con 0x7f71100a2920 2024-03-18T23:08:49.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.065+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 msgr2=0x7f71100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:49.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.065+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 0x7f71100a4de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7114005950 tx=0x7f7114009260 comp rx=0 tx=0).stop 2024-03-18T23:08:49.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.065+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 msgr2=0x7f7124198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:49.066 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.065+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124198a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f710c00e9f0 tx=0x7f710c00ed00 comp rx=0 tx=0).stop 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 shutdown_connections 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f71100a2920 0x7f71100a4de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7124102dd0 0x7f712419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7124101230 0x7f71241984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 --2- 172.21.15.132:0/2457102393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7124102430 0x7f7124198a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.066+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 >> 172.21.15.132:0/2457102393 conn(0x7f71240fc7d0 msgr2=0x7f712406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.067+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 shutdown_connections 2024-03-18T23:08:49.068 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:49.067+0000 7f71290f1700 1 -- 172.21.15.132:0/2457102393 wait complete. 2024-03-18T23:08:49.725 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-18T23:08:49.969 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:50.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:50 smithi132 ceph-mon[32020]: from='client.24977 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:50.366 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:50 smithi132 ceph-mon[32020]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:50.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:50 smithi196 ceph-mon[67089]: from='client.24977 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:50.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:50 smithi196 ceph-mon[67089]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:50 smithi144 ceph-mon[66781]: from='client.24977 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:50 smithi144 ceph-mon[66781]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-18T23:08:51.281 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:51 smithi132 ceph-mon[32020]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-18T23:08:51.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:51 smithi196 ceph-mon[67089]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 -- 172.21.15.132:0/2339518708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 msgr2=0x7f4ecc102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/2339518708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc102760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00099c0 tx=0x7f4ec0009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 -- 172.21.15.132:0/2339518708 shutdown_connections 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/2339518708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103ee0 0x7f4ecc10a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/2339518708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc102760 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/2339518708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103540 0x7f4ecc1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 -- 172.21.15.132:0/2339518708 >> 172.21.15.132:0/2339518708 conn(0x7f4ecc0fd920 msgr2=0x7f4ecc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:51.518 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 -- 172.21.15.132:0/2339518708 shutdown_connections 2024-03-18T23:08:51.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.517+0000 7f4ed2b11700 1 -- 172.21.15.132:0/2339518708 wait complete. 2024-03-18T23:08:51.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 Processor -- start 2024-03-18T23:08:51.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 -- start start 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 0x7f4ecc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103ee0 0x7f4ecc19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10df10 con 0x7f4ecc102340 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10dd90 con 0x7f4ecc103ee0 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed2b11700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10e090 con 0x7f4ecc103540 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed08ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed08ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60558/0 (socket says 172.21.15.132:60558) 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed08ad700 1 -- 172.21.15.132:0/155443444 learned_addr learned my addr 172.21.15.132:0/155443444 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ecbfff700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 0x7f4ecc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.518+0000 7f4ed10ae700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103ee0 0x7f4ecc19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:51.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 -- 172.21.15.132:0/155443444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103ee0 msgr2=0x7f4ecc19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103ee0 0x7f4ecc19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 -- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 msgr2=0x7f4ecc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 -- 172.21.15.132:0/155443444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1a01e0 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ed08ad700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ecbfff700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 0x7f4ecc198ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800ea20 tx=0x7f4eb800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800cb50 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb800ee60 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc0745a0 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8018680 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.519+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc074ae0 con 0x7f4ecc103540 2024-03-18T23:08:51.522 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.520+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc068fd0 con 0x7f4ecc103540 2024-03-18T23:08:51.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.521+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f4eb8047a60 con 0x7f4ecc103540 2024-03-18T23:08:51.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.522+0000 7f4ec9ffb700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 0x7f4ebc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:51.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.522+0000 7f4ed08ad700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 0x7f4ebc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:51.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.522+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4eb80ceae0 con 0x7f4ecc103540 2024-03-18T23:08:51.523 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.522+0000 7f4ed08ad700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 0x7f4ebc0a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:51.526 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.525+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4eb809bb90 con 0x7f4ecc103540 2024-03-18T23:08:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:51 smithi144 ceph-mon[66781]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-03-18T23:08:51.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.730+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4ecc062750 con 0x7f4ebc0a2890 2024-03-18T23:08:51.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.732+0000 7f4ec9ffb700 1 -- 172.21.15.132:0/155443444 <== mgr.14152 v2:172.21.15.132:6800/3730787943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4ecc062750 con 0x7f4ebc0a2890 2024-03-18T23:08:51.732 INFO:teuthology.orchestra.run.smithi132.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-18T23:08:51.732 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-18T23:08:51.732 INFO:teuthology.orchestra.run.smithi132.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T23:08:51.732 INFO:teuthology.orchestra.run.smithi132.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T23:08:51.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 msgr2=0x7f4ebc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:51.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 0x7f4ebc0a4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec0005fb0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 msgr2=0x7f4ecc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 0x7f4ecc198ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800ea20 tx=0x7f4eb800ed30 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 shutdown_connections 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4ebc0a2890 0x7f4ebc0a4d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ecc103ee0 0x7f4ecc19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ecc102340 0x7f4ecc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 --2- 172.21.15.132:0/155443444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ecc103540 0x7f4ecc198ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 >> 172.21.15.132:0/155443444 conn(0x7f4ecc0fd920 msgr2=0x7f4ecc1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 shutdown_connections 2024-03-18T23:08:51.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:51.734+0000 7f4ed2b11700 1 -- 172.21.15.132:0/155443444 wait complete. 2024-03-18T23:08:52.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:52 smithi132 ceph-mon[32020]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:08:52.447 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T23:08:52.457 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:08:52.457 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T23:08:52.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:52 smithi196 ceph-mon[67089]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:08:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:52 smithi144 ceph-mon[66781]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-18T23:08:52.696 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:53.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:53 smithi132 ceph-mon[32020]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:53.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:53 smithi196 ceph-mon[67089]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:53 smithi144 ceph-mon[66781]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 -- 172.21.15.132:0/882639006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468103530 msgr2=0x7f7468103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 --2- 172.21.15.132:0/882639006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468103530 0x7f7468103990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f74580099c0 tx=0x7f7458009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 -- 172.21.15.132:0/882639006 shutdown_connections 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 --2- 172.21.15.132:0/882639006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103ed0 0x7f746810a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 --2- 172.21.15.132:0/882639006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468102330 0x7f7468102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 --2- 172.21.15.132:0/882639006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468103530 0x7f7468103990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.306+0000 7f746f6ab700 1 -- 172.21.15.132:0/882639006 >> 172.21.15.132:0/882639006 conn(0x7f74680fd8f0 msgr2=0x7f74680ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- 172.21.15.132:0/882639006 shutdown_connections 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- 172.21.15.132:0/882639006 wait complete. 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 Processor -- start 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- start start 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103530 0x7f7468198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468103ed0 0x7f746819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f746810df00 con 0x7f7468102330 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f746810dd80 con 0x7f7468103530 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.307+0000 7f746f6ab700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f746810e080 con 0x7f7468103ed0 2024-03-18T23:08:54.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50356/0 (socket says 172.21.15.132:50356) 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 -- 172.21.15.132:0/2001582234 learned_addr learned my addr 172.21.15.132:0/2001582234 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746dc48700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468103ed0 0x7f746819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746cc46700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103530 0x7f7468198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 -- 172.21.15.132:0/2001582234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468103ed0 msgr2=0x7f746819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468103ed0 0x7f746819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 -- 172.21.15.132:0/2001582234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103530 msgr2=0x7f7468198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103530 0x7f7468198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 -- 172.21.15.132:0/2001582234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74680745c0 con 0x7f7468102330 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746d447700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f746400b570 tx=0x7f746400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464004020 con 0x7f7468102330 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7464003680 con 0x7f7468102330 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74680748b0 con 0x7f7468102330 2024-03-18T23:08:54.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7468074e40 con 0x7f7468102330 2024-03-18T23:08:54.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.308+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746401d830 con 0x7f7468102330 2024-03-18T23:08:54.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.309+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7468068fd0 con 0x7f7468102330 2024-03-18T23:08:54.312 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.310+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f7464024020 con 0x7f7468102330 2024-03-18T23:08:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.311+0000 7f745e7fc700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 0x7f74540a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.311+0000 7f746cc46700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 0x7f74540a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.311+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f74640d6470 con 0x7f7468102330 2024-03-18T23:08:54.313 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.311+0000 7f746cc46700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 0x7f74540a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7458000c00 tx=0x7f7458005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:54.316 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.315+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f74640a3520 con 0x7f7468102330 2024-03-18T23:08:54.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:54 smithi132 ceph-mon[32020]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:54.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:54 smithi196 ceph-mon[67089]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:54.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:54 smithi144 ceph-mon[66781]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.558+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f74681093d0 con 0x7f7468102330 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.558+0000 7f745e7fc700 1 -- 172.21.15.132:0/2001582234 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3060 (secure 0 0 0) 0x7f74640a2e20 con 0x7f7468102330 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stdout:e20 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:08:54.559 INFO:teuthology.orchestra.run.smithi132.stdout:legacy client fscid: 1 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:Filesystem 'cephfs' (1) 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:fs_name cephfs 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:epoch 20 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:created 2024-03-18T23:07:13.853025+0000 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:modified 2024-03-18T23:07:18.737695+0000 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:tableserver 0 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:root 0 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:session_timeout 300 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:session_autoclose 300 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:max_file_size 1099511627776 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:max_xattr_size 65536 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:required_client_features {} 2024-03-18T23:08:54.560 INFO:teuthology.orchestra.run.smithi132.stdout:last_failure 0 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:last_failure_osd_epoch 0 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:max_mds 3 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:in 0,1,2 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:up {0=24518,1=24493,2=24485} 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:failed 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:damaged 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:stopped 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:data_pools [3] 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:metadata_pool 2 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:inline_data disabled 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:balancer 2024-03-18T23:08:54.561 INFO:teuthology.orchestra.run.smithi132.stdout:bal_rank_mask -1 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:standby_count_wanted 0 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:qdb_cluster leader: 24518 members: 24493,24518,24485 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b{0:24518} state up:active seq 3 addr [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.196:6836/3483317240,v1:172.21.15.196:6837/3483317240] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.i{1:24493} state up:active seq 5 addr [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.f{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/559973533,v1:172.21.15.196:6835/559973533] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.h{2:24485} state up:active seq 6 addr [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.e{2:24458} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/769602526,v1:172.21.15.144:6835/769602526] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:Standby daemons: 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:54.562 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.132:6834/1067641512,v1:172.21.15.132:6835/1067641512] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.563 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.132:6836/2263462076,v1:172.21.15.132:6837/2263462076] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.563 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.g{-1:14625} state up:standby seq 1 addr [v2:172.21.15.132:6838/4158656425,v1:172.21.15.132:6839/4158656425] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.563 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a{-1:14637} state up:standby seq 1 addr [v2:172.21.15.132:6840/3314453651,v1:172.21.15.132:6841/3314453651] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.563 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.196:6832/3738348032,v1:172.21.15.196:6833/3738348032] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.563 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.k{-1:24422} state up:standby seq 1 addr [v2:172.21.15.144:6832/2374016108,v1:172.21.15.144:6833/2374016108] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.560+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 msgr2=0x7f74540a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.560+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 0x7f74540a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7458000c00 tx=0x7f7458005fb0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.560+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 msgr2=0x7f7468198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.560+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f746400b570 tx=0x7f746400b880 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 shutdown_connections 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f74540a2910 0x7f74540a4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7468103ed0 0x7f746819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7468102330 0x7f7468198550 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 --2- 172.21.15.132:0/2001582234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7468103530 0x7f7468198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 >> 172.21.15.132:0/2001582234 conn(0x7f74680fd8f0 msgr2=0x7f7468106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:54.564 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 shutdown_connections 2024-03-18T23:08:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:54.561+0000 7f746f6ab700 1 -- 172.21.15.132:0/2001582234 wait complete. 2024-03-18T23:08:54.565 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:08:55.310 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:55 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2001582234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:08:55.337 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-18T23:08:55.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:55 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2001582234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:08:55.578 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:08:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:55 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2001582234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T23:08:56.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:56 smithi132 ceph-mon[32020]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-18T23:08:56.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:56 smithi196 ceph-mon[67089]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-18T23:08:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:56 smithi144 ceph-mon[66781]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-18T23:08:57.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 -- 172.21.15.132:0/3935656181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 msgr2=0x7f27301039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 --2- 172.21.15.132:0/3935656181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f27301039a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f27180099c0 tx=0x7f2718009cd0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 -- 172.21.15.132:0/3935656181 shutdown_connections 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 --2- 172.21.15.132:0/3935656181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 0x7f273010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 --2- 172.21.15.132:0/3935656181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 --2- 172.21.15.132:0/3935656181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f27301039a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 -- 172.21.15.132:0/3935656181 >> 172.21.15.132:0/3935656181 conn(0x7f27300fd920 msgr2=0x7f27300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.287+0000 7f27364fd700 1 -- 172.21.15.132:0/3935656181 shutdown_connections 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 -- 172.21.15.132:0/3935656181 wait complete. 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 Processor -- start 2024-03-18T23:08:57.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 -- start start 2024-03-18T23:08:57.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 0x7f273019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.288+0000 7f27364fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f273010df30 con 0x7f2730103540 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f27364fd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f273010ddb0 con 0x7f2730103ee0 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f27364fd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f273010e0b0 con 0x7f2730102340 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50376/0 (socket says 172.21.15.132:50376) 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272f7fe700 1 -- 172.21.15.132:0/1440870740 learned_addr learned my addr 172.21.15.132:0/1440870740 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f2734a9a700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 0x7f273019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 -- 172.21.15.132:0/1440870740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 msgr2=0x7f273019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 0x7f273019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 -- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 msgr2=0x7f2730198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 -- 172.21.15.132:0/1440870740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301a01e0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272f7fe700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272ffff700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730198580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9f0 tx=0x7f272000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720004230 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27200043d0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301a04d0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720003de0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.289+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301a09c0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.290+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2730068fd0 con 0x7f2730102340 2024-03-18T23:08:57.293 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.291+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f272000c7c0 con 0x7f2730102340 2024-03-18T23:08:57.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.292+0000 7f272d7fa700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 0x7f271c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:57.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.292+0000 7f272f7fe700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 0x7f271c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:57.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.292+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f27200cee80 con 0x7f2730102340 2024-03-18T23:08:57.294 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.292+0000 7f272f7fe700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 0x7f271c0a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27180038c0 tx=0x7f2718005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:57.297 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.296+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f272009bf30 con 0x7f2730102340 2024-03-18T23:08:57.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.496+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2730062f20 con 0x7f2730102340 2024-03-18T23:08:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.496+0000 7f272d7fa700 1 -- 172.21.15.132:0/1440870740 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5306 (secure 0 0 0) 0x7f272009b830 con 0x7f2730102340 2024-03-18T23:08:57.497 INFO:teuthology.orchestra.run.smithi132.stdout:epoch 90 2024-03-18T23:08:57.497 INFO:teuthology.orchestra.run.smithi132.stdout:fsid 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:08:57.497 INFO:teuthology.orchestra.run.smithi132.stdout:created 2024-03-18T22:59:19.051809+0000 2024-03-18T23:08:57.497 INFO:teuthology.orchestra.run.smithi132.stdout:modified 2024-03-18T23:08:20.188080+0000 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:crush_version 26 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:full_ratio 0.95 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:backfillfull_ratio 0.9 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:nearfull_ratio 0.85 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:require_min_compat_client luminous 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:min_compat_client luminous 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:require_osd_release squid 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:stretch_mode_enabled false 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-18T23:08:57.498 INFO:teuthology.orchestra.run.smithi132.stdout:max_osd 12 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6802/3523914432,v1:172.21.15.132:6803/3523914432] [v2:172.21.15.132:6804/3523914432,v1:172.21.15.132:6805/3523914432] exists,up f8845c7f-34df-49a6-96ee-d71d153dc0f2 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/1173848453,v1:172.21.15.144:6801/1173848453] [v2:172.21.15.144:6802/1173848453,v1:172.21.15.144:6803/1173848453] exists,up 3e2cb994-9178-41e5-9724-62360d566915 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/1879525213,v1:172.21.15.196:6801/1879525213] [v2:172.21.15.196:6802/1879525213,v1:172.21.15.196:6803/1879525213] exists,up 2adeb4b3-32ca-446e-9801-d535dd74c2d9 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6810/3584795430,v1:172.21.15.132:6811/3584795430] [v2:172.21.15.132:6812/3584795430,v1:172.21.15.132:6813/3584795430] exists,up 253c2374-3879-4485-8ecb-a893a210b624 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/962231389,v1:172.21.15.144:6809/962231389] [v2:172.21.15.144:6810/962231389,v1:172.21.15.144:6811/962231389] exists,up 5e8564c1-d17a-453d-a5e2-af4740fb0d93 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/1616570087,v1:172.21.15.196:6809/1616570087] [v2:172.21.15.196:6810/1616570087,v1:172.21.15.196:6811/1616570087] exists,up 5eedd04e-128b-491d-82e6-1988d13b1145 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.6 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6818/428682768,v1:172.21.15.132:6819/428682768] [v2:172.21.15.132:6820/428682768,v1:172.21.15.132:6821/428682768] exists,up df2ea776-34b0-45ca-8bfa-f7e5428ce0c9 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.7 up in weight 1 up_from 54 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/3972442848,v1:172.21.15.144:6817/3972442848] [v2:172.21.15.144:6818/3972442848,v1:172.21.15.144:6819/3972442848] exists,up cf357f13-d06f-49c0-b598-1cadcd305fa3 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.8 up in weight 1 up_from 62 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3559751523,v1:172.21.15.196:6817/3559751523] [v2:172.21.15.196:6818/3559751523,v1:172.21.15.196:6819/3559751523] exists,up a8d245b3-4a54-4056-92f6-0c30db4759cb 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.9 up in weight 1 up_from 70 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6826/118980095,v1:172.21.15.132:6827/118980095] [v2:172.21.15.132:6828/118980095,v1:172.21.15.132:6829/118980095] exists,up 5b2f5b52-8981-48f2-9229-27493b6d62f2 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.10 up in weight 1 up_from 76 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/2873915844,v1:172.21.15.144:6825/2873915844] [v2:172.21.15.144:6826/2873915844,v1:172.21.15.144:6827/2873915844] exists,up 5922d4cd-937c-44ec-b82c-7fec2f16c8c4 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/946882152,v1:172.21.15.196:6825/946882152] [v2:172.21.15.196:6826/946882152,v1:172.21.15.196:6827/946882152] exists,up d26113fc-541e-4090-89e8-909b4c0d5d05 2024-03-18T23:08:57.499 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.4 [2,11] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.5 [8,11] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 2.c [2,4] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.11 [4,8] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.14 [4,6] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.33 [5,9] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.38 [3,9] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:pg_upmap_items 3.3c [4,9] 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/1441879282 expires 2024-03-19T23:00:18.961831+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/2712624534 expires 2024-03-19T23:00:18.961831+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6801/3371561746 expires 2024-03-19T23:00:18.961831+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6801/1223581697 expires 2024-03-19T22:59:52.674103+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/2884383249 expires 2024-03-19T22:59:52.674103+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6800/3371561746 expires 2024-03-19T23:00:18.961831+0000 2024-03-18T23:08:57.500 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:6800/1223581697 expires 2024-03-19T22:59:52.674103+0000 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/4178269826 expires 2024-03-19T22:59:52.674103+0000 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/3720862041 expires 2024-03-19T22:59:52.674103+0000 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stdout:blocklist 172.21.15.132:0/175028242 expires 2024-03-19T23:00:18.961831+0000 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 msgr2=0x7f271c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 0x7f271c0a4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27180038c0 tx=0x7f2718005fb0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 msgr2=0x7f2730198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730198580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9f0 tx=0x7f272000ed00 comp rx=0 tx=0).stop 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 shutdown_connections 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f271c0a2890 0x7f271c0a4d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730103ee0 0x7f273019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2730102340 0x7f2730198580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 --2- 172.21.15.132:0/1440870740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2730103540 0x7f2730198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 >> 172.21.15.132:0/1440870740 conn(0x7f27300fd920 msgr2=0x7f27301068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 shutdown_connections 2024-03-18T23:08:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:57.499+0000 7f27364fd700 1 -- 172.21.15.132:0/1440870740 wait complete. 2024-03-18T23:08:58.152 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-18T23:08:58.225 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 -- 172.21.15.132:0/2774900441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67800ffb10 msgr2=0x7f67800fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 --2- 172.21.15.132:0/2774900441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67800ffb10 0x7f67800fff30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6770009930 tx=0x7f677002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 -- 172.21.15.132:0/2774900441 shutdown_connections 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 --2- 172.21.15.132:0/2774900441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 0x7f6780109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 --2- 172.21.15.132:0/2774900441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6780102ae0 0x7f6780102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 --2- 172.21.15.132:0/2774900441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67800ffb10 0x7f67800fff30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 -- 172.21.15.132:0/2774900441 >> 172.21.15.132:0/2774900441 conn(0x7f67800fb400 msgr2=0x7f67800fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 -- 172.21.15.132:0/2774900441 shutdown_connections 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.331+0000 7f67884f8640 1 -- 172.21.15.132:0/2774900441 wait complete. 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 Processor -- start 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 -- start start 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67800ffb10 0x7f6780197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 0x7f678019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f678010d4c0 con 0x7f6780102ae0 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f678010d340 con 0x7f67801034a0 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f67884f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f678010d640 con 0x7f67800ffb10 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50392/0 (socket says 172.21.15.132:50392) 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 -- 172.21.15.132:0/531100321 learned_addr learned my addr 172.21.15.132:0/531100321 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6786a95640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 0x7f678019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6786294640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67800ffb10 0x7f6780197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 -- 172.21.15.132:0/531100321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67800ffb10 msgr2=0x7f6780197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67800ffb10 0x7f6780197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 -- 172.21.15.132:0/531100321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 msgr2=0x7f678019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 0x7f678019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.332+0000 7f6785a93640 1 -- 172.21.15.132:0/531100321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678019f7c0 con 0x7f6780102ae0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f6785a93640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e9f0 tx=0x7f677c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00ccc0 con 0x7f6780102ae0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678019fab0 con 0x7f6780102ae0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c010910 con 0x7f6780102ae0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0188a0 con 0x7f6780102ae0 2024-03-18T23:08:58.336 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.333+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67801a0050 con 0x7f6780102ae0 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.334+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6780108d80 con 0x7f6780102ae0 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.335+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f677c010ac0 con 0x7f6780102ae0 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.335+0000 7f67777fe640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 0x7f676c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.336+0000 7f6786294640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 0x7f676c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.336+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f677c0cee90 con 0x7f6780102ae0 2024-03-18T23:08:58.337 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.336+0000 7f6786294640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 0x7f676c0a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f677002c7b0 tx=0x7f6770000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.340+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f677c09bf40 con 0x7f6780102ae0 2024-03-18T23:08:58.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:58 smithi196 ceph-mon[67089]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:58.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:58 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1440870740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:08:58.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.515+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6780062420 con 0x7f6780102ae0 2024-03-18T23:08:58.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.517+0000 7f67777fe640 1 -- 172.21.15.132:0/531100321 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7f677c09b840 con 0x7f6780102ae0 2024-03-18T23:08:58.518 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:58.518 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 msgr2=0x7f676c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 0x7f676c0a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f677002c7b0 tx=0x7f6770000bc0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 msgr2=0x7f67801980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e9f0 tx=0x7f677c00edf0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 shutdown_connections 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f676c0a2800 0x7f676c0a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.520 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67801034a0 0x7f678019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6780102ae0 0x7f67801980e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 --2- 172.21.15.132:0/531100321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67800ffb10 0x7f6780197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 >> 172.21.15.132:0/531100321 conn(0x7f67800fb400 msgr2=0x7f6780105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:58.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 shutdown_connections 2024-03-18T23:08:58.521 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.519+0000 7f67884f8640 1 -- 172.21.15.132:0/531100321 wait complete. 2024-03-18T23:08:58.530 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:58 smithi132 ceph-mon[32020]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:58.530 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:58 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1440870740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:08:58.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:08:58.532 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:08:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:58 smithi144 ceph-mon[66781]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:08:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:58 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1440870740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T23:08:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- 172.21.15.132:0/1276560791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1960100740 msgr2=0x7f1960100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 --2- 172.21.15.132:0/1276560791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1960100740 0x7f1960100b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1950009900 tx=0x7f195002bc30 comp rx=0 tx=0).stop 2024-03-18T23:08:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- 172.21.15.132:0/1276560791 shutdown_connections 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 --2- 172.21.15.132:0/1276560791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19601022f0 0x7f1960108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 --2- 172.21.15.132:0/1276560791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f1960101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 --2- 172.21.15.132:0/1276560791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1960100740 0x7f1960100b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- 172.21.15.132:0/1276560791 >> 172.21.15.132:0/1276560791 conn(0x7f19600fbcd0 msgr2=0x7f19600fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- 172.21.15.132:0/1276560791 shutdown_connections 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- 172.21.15.132:0/1276560791 wait complete. 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 Processor -- start 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.636+0000 7f19666f7640 1 -- start start 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1960100740 0x7f1960197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19601022f0 0x7f196019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d4e0 con 0x7f1960101930 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d360 con 0x7f1960100740 2024-03-18T23:08:58.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f196010d660 con 0x7f19601022f0 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50414/0 (socket says 172.21.15.132:50414) 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 -- 172.21.15.132:0/2639361034 learned_addr learned my addr 172.21.15.132:0/2639361034 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1960100740 0x7f1960197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f1964c94640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19601022f0 0x7f196019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 -- 172.21.15.132:0/2639361034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19601022f0 msgr2=0x7f196019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19601022f0 0x7f196019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 -- 172.21.15.132:0/2639361034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1960100740 msgr2=0x7f1960197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1960100740 0x7f1960197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 -- 172.21.15.132:0/2639361034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196019f780 con 0x7f1960101930 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195f7fe640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9f0 tx=0x7f195400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195400cd80 con 0x7f1960101930 2024-03-18T23:08:58.640 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196019fa70 con 0x7f1960101930 2024-03-18T23:08:58.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1954010910 con 0x7f1960101930 2024-03-18T23:08:58.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540188f0 con 0x7f1960101930 2024-03-18T23:08:58.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.637+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a0010 con 0x7f1960101930 2024-03-18T23:08:58.641 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.638+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1960107a60 con 0x7f1960101930 2024-03-18T23:08:58.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.639+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f1954010ac0 con 0x7f1960101930 2024-03-18T23:08:58.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.640+0000 7f195d7fa640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.640+0000 7f195ffff640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.640+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f19540cee10 con 0x7f1960101930 2024-03-18T23:08:58.642 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.640+0000 7f195ffff640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 0x7f19480a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1950000c00 tx=0x7f1950005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.645 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.645+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f195409b870 con 0x7f1960101930 2024-03-18T23:08:58.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.820+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f196002d1f0 con 0x7f1960101930 2024-03-18T23:08:58.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.820+0000 7f195d7fa640 1 -- 172.21.15.132:0/2639361034 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f195409f910 con 0x7f1960101930 2024-03-18T23:08:58.821 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:08:58.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.822+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 msgr2=0x7f19480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.822+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 0x7f19480a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1950000c00 tx=0x7f1950005fd0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.822+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 msgr2=0x7f19601980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.822+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f195400e9f0 tx=0x7f195400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 shutdown_connections 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f19480a2800 0x7f19480a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19601022f0 0x7f196019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1960101930 0x7f19601980a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 --2- 172.21.15.132:0/2639361034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1960100740 0x7f1960197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 >> 172.21.15.132:0/2639361034 conn(0x7f19600fbcd0 msgr2=0x7f1960104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 shutdown_connections 2024-03-18T23:08:58.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.823+0000 7f19666f7640 1 -- 172.21.15.132:0/2639361034 wait complete. 2024-03-18T23:08:58.835 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 -- 172.21.15.132:0/3463834413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784101840 msgr2=0x7f4784101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 --2- 172.21.15.132:0/3463834413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784101840 0x7f4784101c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f478000fb60 tx=0x7f478002fcf0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 -- 172.21.15.132:0/3463834413 shutdown_connections 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 --2- 172.21.15.132:0/3463834413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f4784109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 --2- 172.21.15.132:0/3463834413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784102a30 0x7f4784102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 --2- 172.21.15.132:0/3463834413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784101840 0x7f4784101c60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 -- 172.21.15.132:0/3463834413 >> 172.21.15.132:0/3463834413 conn(0x7f47840fcd90 msgr2=0x7f47840ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 -- 172.21.15.132:0/3463834413 shutdown_connections 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.938+0000 7f478ba4b640 1 -- 172.21.15.132:0/3463834413 wait complete. 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 Processor -- start 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 -- start start 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 0x7f4784195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784102a30 0x7f4784195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f478419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f478410b2e0 con 0x7f47841033f0 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f478410b160 con 0x7f4784101840 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f478410b460 con 0x7f4784102a30 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f478419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f47897e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 0x7f4784195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f47897e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 0x7f4784195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38208/0 (socket says 172.21.15.132:38208) 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f47897e7640 1 -- 172.21.15.132:0/1283862212 learned_addr learned my addr 172.21.15.132:0/1283862212 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4788fe6640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784102a30 0x7f4784195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 -- 172.21.15.132:0/1283862212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784102a30 msgr2=0x7f4784195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784102a30 0x7f4784195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 -- 172.21.15.132:0/1283862212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 msgr2=0x7f4784195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 0x7f4784195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 -- 172.21.15.132:0/1283862212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478419d590 con 0x7f47841033f0 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f4789fe8640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f478419cf20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f477c00efc0 tx=0x7f477c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c003970 con 0x7f47841033f0 2024-03-18T23:08:58.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f477c004020 con 0x7f47841033f0 2024-03-18T23:08:58.944 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c00b9f0 con 0x7f47841033f0 2024-03-18T23:08:58.944 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.939+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478419d880 con 0x7f47841033f0 2024-03-18T23:08:58.944 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.940+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478419dd80 con 0x7f47841033f0 2024-03-18T23:08:58.944 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.940+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784108be0 con 0x7f47841033f0 2024-03-18T23:08:58.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.941+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f477c01d020 con 0x7f47841033f0 2024-03-18T23:08:58.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.942+0000 7f477a7fc640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 0x7f47700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:58.945 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.942+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f477c0ce8f0 con 0x7f47841033f0 2024-03-18T23:08:58.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.942+0000 7f47897e7640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 0x7f47700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:58.946 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.942+0000 7f47897e7640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 0x7f47700a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f478000f8f0 tx=0x7f4780038be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:58.947 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:58.947+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f477c09b9a0 con 0x7f47841033f0 2024-03-18T23:08:59.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.123+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4784062420 con 0x7f47841033f0 2024-03-18T23:08:59.124 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.124+0000 7f477a7fc640 1 -- 172.21.15.132:0/1283862212 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f477c09b2a0 con 0x7f47841033f0 2024-03-18T23:08:59.124 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:08:59.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 msgr2=0x7f47700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.126 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 0x7f47700a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f478000f8f0 tx=0x7f4780038be0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 msgr2=0x7f478419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f478419cf20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f477c00efc0 tx=0x7f477c00c800 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 shutdown_connections 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f47700a2800 0x7f47700a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47841033f0 0x7f478419cf20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4784102a30 0x7f4784195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 --2- 172.21.15.132:0/1283862212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4784101840 0x7f4784195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 >> 172.21.15.132:0/1283862212 conn(0x7f47840fcd90 msgr2=0x7f4784105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 shutdown_connections 2024-03-18T23:08:59.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.126+0000 7f478ba4b640 1 -- 172.21.15.132:0/1283862212 wait complete. 2024-03-18T23:08:59.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:08:59.244 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/2202238794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe40ffe70 msgr2=0x7fbbe4100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/2202238794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4100290 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800aa10 tx=0x7fbbd802cf70 comp rx=0 tx=0).stop 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/2202238794 shutdown_connections 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/2202238794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe4101a10 0x7fbbe4108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/2202238794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101050 0x7fbbe41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/2202238794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4100290 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/2202238794 >> 172.21.15.132:0/2202238794 conn(0x7fbbe40fb450 msgr2=0x7fbbe40fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/2202238794 shutdown_connections 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/2202238794 wait complete. 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 Processor -- start 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.242+0000 7fbbeaa7e640 1 -- start start 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101a10 0x7fbbe419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe410b320 con 0x7fbbe4101050 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe410b1a0 con 0x7fbbe40ffe70 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe410b4a0 con 0x7fbbe4101a10 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50448/0 (socket says 172.21.15.132:50448) 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 -- 172.21.15.132:0/1959142668 learned_addr learned my addr 172.21.15.132:0/1959142668 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe881a640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe901b640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101a10 0x7fbbe419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 -- 172.21.15.132:0/1959142668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101a10 msgr2=0x7fbbe419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101a10 0x7fbbe419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 -- 172.21.15.132:0/1959142668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 msgr2=0x7fbbe4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 -- 172.21.15.132:0/1959142668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe419f7c0 con 0x7fbbe4101050 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe881a640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe3fff640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9f0 tx=0x7fbbd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000cd80 con 0x7fbbe4101050 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd0005500 con 0x7fbbe4101050 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe419fab0 con 0x7fbbe4101050 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.243+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0010690 con 0x7fbbe4101050 2024-03-18T23:08:59.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.244+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe419ffb0 con 0x7fbbe4101050 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.245+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe4107210 con 0x7fbbe4101050 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.245+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fbbd00046c0 con 0x7fbbe4101050 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.246+0000 7fbbe1ffb640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 0x7fbbd40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.246+0000 7fbbe881a640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 0x7fbbd40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.246+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbbd00cee40 con 0x7fbbe4101050 2024-03-18T23:08:59.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.246+0000 7fbbe881a640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 0x7fbbd40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbbd802d4b0 tx=0x7fbbd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.251 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.250+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbbd009bef0 con 0x7fbbe4101050 2024-03-18T23:08:59.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.424+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbbe402d1f0 con 0x7fbbe4101050 2024-03-18T23:08:59.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.424+0000 7fbbe1ffb640 1 -- 172.21.15.132:0/1959142668 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fbbd009b7f0 con 0x7fbbe4101050 2024-03-18T23:08:59.425 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:08:59.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 msgr2=0x7fbbd40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 0x7fbbd40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbbd802d4b0 tx=0x7fbbd8000bc0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 msgr2=0x7fbbe41980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e9f0 tx=0x7fbbd000edf0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 shutdown_connections 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fbbd40a2800 0x7fbbd40a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbe4101a10 0x7fbbe419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe4101050 0x7fbbe41980e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 --2- 172.21.15.132:0/1959142668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe40ffe70 0x7fbbe4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.426+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 >> 172.21.15.132:0/1959142668 conn(0x7fbbe40fb450 msgr2=0x7fbbe41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.427+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 shutdown_connections 2024-03-18T23:08:59.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.427+0000 7fbbeaa7e640 1 -- 172.21.15.132:0/1959142668 wait complete. 2024-03-18T23:08:59.437 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:59 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/531100321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:59.437 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:59 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2639361034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.437 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:08:59 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1283862212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:08:59.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:59 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/531100321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:59.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:59 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2639361034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:08:59 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1283862212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 -- 172.21.15.132:0/1567707578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa81033f0 msgr2=0x7f4aa8109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.541 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/1567707578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa81033f0 0x7f4aa8109c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009be0 tx=0x7f4aa002bef0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 -- 172.21.15.132:0/1567707578 shutdown_connections 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/1567707578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa81033f0 0x7f4aa8109c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/1567707578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/1567707578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa8101840 0x7f4aa8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 -- 172.21.15.132:0/1567707578 >> 172.21.15.132:0/1567707578 conn(0x7f4aa80fcdd0 msgr2=0x7f4aa80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 -- 172.21.15.132:0/1567707578 shutdown_connections 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.539+0000 7f4aaedf3640 1 -- 172.21.15.132:0/1567707578 wait complete. 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 Processor -- start 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 -- start start 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 0x7f4aa8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.542 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa81033f0 0x7f4aa819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa810d4a0 con 0x7f4aa8102a30 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa810d320 con 0x7f4aa81033f0 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa810d620 con 0x7f4aa8101840 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50460/0 (socket says 172.21.15.132:50460) 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 -- 172.21.15.132:0/3590189231 learned_addr learned my addr 172.21.15.132:0/3590189231 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aacb8f640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 0x7f4aa8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aad390640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa81033f0 0x7f4aa819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 -- 172.21.15.132:0/3590189231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 msgr2=0x7f4aa8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 0x7f4aa8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 -- 172.21.15.132:0/3590189231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa81033f0 msgr2=0x7f4aa819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa81033f0 0x7f4aa819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 -- 172.21.15.132:0/3590189231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa819f730 con 0x7f4aa8102a30 2024-03-18T23:08:59.543 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aacb8f640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 0x7f4aa8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa7fff640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400e9c0 tx=0x7f4a9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400cde0 con 0x7f4aa8102a30 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa819fa20 con 0x7f4aa8102a30 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a940035f0 con 0x7f4aa8102a30 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.540+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94010650 con 0x7f4aa8102a30 2024-03-18T23:08:59.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.541+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa819ff70 con 0x7f4aa8102a30 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.542+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa8108b00 con 0x7f4aa8102a30 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.543+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f4a940037a0 con 0x7f4aa8102a30 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.543+0000 7f4aa5ffb640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 0x7f4a980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.543+0000 7f4aacb8f640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 0x7f4a980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.543+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4a940ced50 con 0x7f4aa8102a30 2024-03-18T23:08:59.545 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.544+0000 7f4aacb8f640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 0x7f4a980a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8102890 tx=0x7f4a9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.548 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.548+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a940d2030 con 0x7f4aa8102a30 2024-03-18T23:08:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:59 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/531100321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:08:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:59 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2639361034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:08:59 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1283862212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:08:59.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.759+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa8062420 con 0x7f4aa8102a30 2024-03-18T23:08:59.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.760+0000 7f4aa5ffb640 1 -- 172.21.15.132:0/3590189231 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7f4a9409be00 con 0x7f4aa8102a30 2024-03-18T23:08:59.761 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:08:59.761 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:08:59.761 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 msgr2=0x7f4a980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 0x7f4a980a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8102890 tx=0x7f4a9c005f90 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 msgr2=0x7f4aa81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400e9c0 tx=0x7f4a9400ee80 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 shutdown_connections 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f4a980a2800 0x7f4a980a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4aa81033f0 0x7f4aa819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aa8102a30 0x7f4aa81980f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 --2- 172.21.15.132:0/3590189231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4aa8101840 0x7f4aa8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 >> 172.21.15.132:0/3590189231 conn(0x7f4aa80fcdd0 msgr2=0x7f4aa8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 shutdown_connections 2024-03-18T23:08:59.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.762+0000 7f4aaedf3640 1 -- 172.21.15.132:0/3590189231 wait complete. 2024-03-18T23:08:59.774 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:08:59.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/2229816213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 msgr2=0x7fa7b8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/2229816213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b8108c00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009ca0 tx=0x7fa7b002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/2229816213 shutdown_connections 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/2229816213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b8108c00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/2229816213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81019b0 0x7fa7b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/2229816213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81007c0 0x7fa7b8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/2229816213 >> 172.21.15.132:0/2229816213 conn(0x7fa7b80fbd50 msgr2=0x7fa7b80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/2229816213 shutdown_connections 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/2229816213 wait complete. 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 Processor -- start 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.875+0000 7fa7bf0bf640 1 -- start start 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81007c0 0x7fa7b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.878 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81019b0 0x7fa7b8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c3b0 con 0x7fa7b8102370 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c230 con 0x7fa7b81007c0 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c530 con 0x7fa7b81019b0 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50476/0 (socket says 172.21.15.132:50476) 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 -- 172.21.15.132:0/3801768873 learned_addr learned my addr 172.21.15.132:0/3801768873 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bce5b640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81007c0 0x7fa7b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7b7fff640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81019b0 0x7fa7b8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 -- 172.21.15.132:0/3801768873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81019b0 msgr2=0x7fa7b8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81019b0 0x7fa7b8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 -- 172.21.15.132:0/3801768873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81007c0 msgr2=0x7fa7b8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:08:59.879 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81007c0 0x7fa7b8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 -- 172.21.15.132:0/3801768873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b819f710 con 0x7fa7b8102370 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bd65c640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0037040 tx=0x7fa7b0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0041020 con 0x7fa7b8102370 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b0034cd0 con 0x7fa7b8102370 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819f9a0 con 0x7fa7b8102370 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0039040 con 0x7fa7b8102370 2024-03-18T23:08:59.880 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.876+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b819fee0 con 0x7fa7b8102370 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.877+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b81a0110 con 0x7fa7b8102370 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.879+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fa7b0044020 con 0x7fa7b8102370 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.879+0000 7fa7b5ffb640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 0x7fa7a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.879+0000 7fa7bce5b640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 0x7fa7a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.879+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fa7b00f0c50 con 0x7fa7b8102370 2024-03-18T23:08:59.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.879+0000 7fa7bce5b640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 0x7fa7a80a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0095f0 tx=0x7fa7ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:08:59.883 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:08:59.883+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7b004b020 con 0x7fa7b8102370 2024-03-18T23:09:00.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.057+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7b81a04a0 con 0x7fa7b8102370 2024-03-18T23:09:00.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.058+0000 7fa7b5ffb640 1 -- 172.21.15.132:0/3801768873 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7fa7b00bdd90 con 0x7fa7b8102370 2024-03-18T23:09:00.059 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:09:00.060 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 msgr2=0x7fa7a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 0x7fa7a80a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0095f0 tx=0x7fa7ac005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 msgr2=0x7fa7b819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0037040 tx=0x7fa7b0032f40 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 shutdown_connections 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fa7a80a2880 0x7fa7a80a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7b8102370 0x7fa7b819f0a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b81019b0 0x7fa7b8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 --2- 172.21.15.132:0/3801768873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7b81007c0 0x7fa7b8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 >> 172.21.15.132:0/3801768873 conn(0x7fa7b80fbd50 msgr2=0x7fa7b80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 shutdown_connections 2024-03-18T23:09:00.062 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.061+0000 7fa7bf0bf640 1 -- 172.21.15.132:0/3801768873 wait complete. 2024-03-18T23:09:00.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:09:00.074 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:09:00.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- 172.21.15.132:0/2264621645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 msgr2=0x7fec0c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- 172.21.15.132:0/2264621645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c101d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec08009860 tx=0x7fec0802bb70 comp rx=0 tx=0).stop 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- 172.21.15.132:0/2264621645 shutdown_connections 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- 172.21.15.132:0/2264621645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c1022d0 0x7fec0c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- 172.21.15.132:0/2264621645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c101d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- 172.21.15.132:0/2264621645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c100720 0x7fec0c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- 172.21.15.132:0/2264621645 >> 172.21.15.132:0/2264621645 conn(0x7fec0c0fbcb0 msgr2=0x7fec0c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- 172.21.15.132:0/2264621645 shutdown_connections 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- 172.21.15.132:0/2264621645 wait complete. 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 Processor -- start 2024-03-18T23:09:00.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- start start 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c100720 0x7fec0c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10c3a0 con 0x7fec0c1022d0 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10c220 con 0x7fec0c100720 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.178+0000 7fec13b77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10c520 con 0x7fec0c101910 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec12114640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:36668/0 (socket says 172.21.15.132:36668) 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec12114640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50500/0 (socket says 172.21.15.132:50500) 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 -- 172.21.15.132:0/3072692414 learned_addr learned my addr 172.21.15.132:0/3072692414 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:00.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11913640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c100720 0x7fec0c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 -- 172.21.15.132:0/3072692414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c100720 msgr2=0x7fec0c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c100720 0x7fec0c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 -- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 msgr2=0x7fec0c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 -- 172.21.15.132:0/3072692414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c19f7b0 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec12114640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec11112640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec08005940 tx=0x7fec08032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08041020 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0802c9f0 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c102b30 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08039040 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.179+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c103020 con 0x7fec0c101910 2024-03-18T23:09:00.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.180+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c103280 con 0x7fec0c101910 2024-03-18T23:09:00.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.181+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fec08044020 con 0x7fec0c101910 2024-03-18T23:09:00.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.182+0000 7fec02ffd640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 0x7febf80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.182+0000 7fec11913640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 0x7febf80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.182+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fec080f07b0 con 0x7fec0c101910 2024-03-18T23:09:00.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.182+0000 7fec11913640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 0x7febf80a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec0c101770 tx=0x7febfc012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.185+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec0804b020 con 0x7fec0c101910 2024-03-18T23:09:00.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.359+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec0c1036d0 con 0x7fec0c101910 2024-03-18T23:09:00.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.360+0000 7fec02ffd640 1 -- 172.21.15.132:0/3072692414 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fec080bd860 con 0x7fec0c101910 2024-03-18T23:09:00.361 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:09:00.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 msgr2=0x7febf80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 0x7febf80a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec0c101770 tx=0x7febfc012040 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 msgr2=0x7fec0c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec08005940 tx=0x7fec08032470 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 shutdown_connections 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7febf80a2800 0x7febf80a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec0c1022d0 0x7fec0c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec0c101910 0x7fec0c1980d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 --2- 172.21.15.132:0/3072692414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec0c100720 0x7fec0c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 >> 172.21.15.132:0/3072692414 conn(0x7fec0c0fbcb0 msgr2=0x7fec0c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 shutdown_connections 2024-03-18T23:09:00.363 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.362+0000 7fec13b77640 1 -- 172.21.15.132:0/3072692414 wait complete. 2024-03-18T23:09:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:00 smithi132 ceph-mon[32020]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-18T23:09:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1959142668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3590189231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:00 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3801768873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:09:00.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:00 smithi196 ceph-mon[67089]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-18T23:09:00.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1959142668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:00.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3590189231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:00 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3801768873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- 172.21.15.132:0/3787833161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4104c50 msgr2=0x7f90d4107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3787833161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4104c50 0x7f90d4107040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009900 tx=0x7f90c002bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:00.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- 172.21.15.132:0/3787833161 shutdown_connections 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3787833161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4107580 0x7f90d4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3787833161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4104c50 0x7f90d4107040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3787833161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4102320 0x7f90d4104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- 172.21.15.132:0/3787833161 >> 172.21.15.132:0/3787833161 conn(0x7f90d40fbcd0 msgr2=0x7f90d40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- 172.21.15.132:0/3787833161 shutdown_connections 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- 172.21.15.132:0/3787833161 wait complete. 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 Processor -- start 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 -- start start 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4102320 0x7f90d4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.478+0000 7f90dabe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90dabe6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90dabe6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410d4c0 con 0x7f90d4107580 2024-03-18T23:09:00.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90dabe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410d340 con 0x7f90d4104c50 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90dabe6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410d640 con 0x7f90d4102320 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d9183640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d8982640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4102320 0x7f90d4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38286/0 (socket says 172.21.15.132:38286) 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 -- 172.21.15.132:0/3804525413 learned_addr learned my addr 172.21.15.132:0/3804525413 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d9183640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50522/0 (socket says 172.21.15.132:50522) 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 -- 172.21.15.132:0/3804525413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4102320 msgr2=0x7f90d4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4102320 0x7f90d4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 -- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 msgr2=0x7f90d419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 -- 172.21.15.132:0/3804525413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d419f7b0 con 0x7f90d4104c50 2024-03-18T23:09:00.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d9183640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d3fff640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0041070 con 0x7f90d4104c50 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d419fa40 con 0x7f90d4104c50 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c0004450 con 0x7f90d4104c50 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.479+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0034440 con 0x7f90d4104c50 2024-03-18T23:09:00.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.480+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d419ff30 con 0x7f90d4104c50 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.480+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d40fd7c0 con 0x7f90d4104c50 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.481+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f90c0044070 con 0x7f90d4104c50 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.482+0000 7f90d1ffb640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 0x7f90c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.482+0000 7f90d8982640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 0x7f90c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.482+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f90c00f0740 con 0x7f90d4104c50 2024-03-18T23:09:00.484 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.482+0000 7f90d8982640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 0x7f90c40a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90c80095f0 tx=0x7f90c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.490+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f90c00bd1d0 con 0x7f90d4104c50 2024-03-18T23:09:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:00 smithi144 ceph-mon[66781]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-18T23:09:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1959142668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3590189231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:00 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3801768873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:00.664 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.664+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90d402d440 con 0x7f90d4104c50 2024-03-18T23:09:00.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.664+0000 7f90d1ffb640 1 -- 172.21.15.132:0/3804525413 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f90c00c1270 con 0x7f90d4104c50 2024-03-18T23:09:00.665 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 msgr2=0x7f90c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 0x7f90c40a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90c80095f0 tx=0x7f90c8005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 msgr2=0x7f90d41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c002c640 comp rx=0 tx=0).stop 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 shutdown_connections 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f90c40a2800 0x7f90c40a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90d4107580 0x7f90d419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4104c50 0x7f90d41980d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 --2- 172.21.15.132:0/3804525413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90d4102320 0x7f90d4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 >> 172.21.15.132:0/3804525413 conn(0x7f90d40fbcd0 msgr2=0x7f90d4102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 shutdown_connections 2024-03-18T23:09:00.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.666+0000 7f90dabe6640 1 -- 172.21.15.132:0/3804525413 wait complete. 2024-03-18T23:09:00.678 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:09:00.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.782+0000 7f6d083c6640 1 -- 172.21.15.132:0/1306248314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00101850 msgr2=0x7f6d00101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.782+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1306248314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00101850 0x7f6d00101c70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009ca0 tx=0x7f6cfc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.785 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.782+0000 7f6d083c6640 1 -- 172.21.15.132:0/1306248314 shutdown_connections 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.782+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1306248314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00103400 0x7f6d00109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1306248314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00102a40 0x7f6d00102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1306248314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00101850 0x7f6d00101c70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- 172.21.15.132:0/1306248314 >> 172.21.15.132:0/1306248314 conn(0x7f6d000fcdc0 msgr2=0x7f6d000ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- 172.21.15.132:0/1306248314 shutdown_connections 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- 172.21.15.132:0/1306248314 wait complete. 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 Processor -- start 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- start start 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 0x7f6d00197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00102a40 0x7f6d00198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.786 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d4a0 con 0x7f6d00103400 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d320 con 0x7f6d00102a40 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d083c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d620 con 0x7f6d00101850 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d06963640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d06963640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50530/0 (socket says 172.21.15.132:50530) 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d06963640 1 -- 172.21.15.132:0/1359479563 learned_addr learned my addr 172.21.15.132:0/1359479563 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d06162640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 0x7f6d00197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.783+0000 7f6d05961640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00102a40 0x7f6d00198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 -- 172.21.15.132:0/1359479563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 msgr2=0x7f6d00197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 0x7f6d00197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 -- 172.21.15.132:0/1359479563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00102a40 msgr2=0x7f6d00198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00102a40 0x7f6d00198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 -- 172.21.15.132:0/1359479563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0019f760 con 0x7f6d00103400 2024-03-18T23:09:00.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06162640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 0x7f6d00197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d06963640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800e970 tx=0x7f6cf800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800ccc0 con 0x7f6d00103400 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf800ce60 con 0x7f6d00103400 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0019fa50 con 0x7f6d00103400 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8010490 con 0x7f6d00103400 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.784+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0019ff50 con 0x7f6d00103400 2024-03-18T23:09:00.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.785+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d00108bf0 con 0x7f6d00103400 2024-03-18T23:09:00.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.786+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6cf8005020 con 0x7f6d00103400 2024-03-18T23:09:00.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.786+0000 7f6cf77fe640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 0x7f6cec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:00.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.786+0000 7f6d06162640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 0x7f6cec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:00.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.786+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f6cf80ceed0 con 0x7f6d00103400 2024-03-18T23:09:00.789 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.787+0000 7f6d06162640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 0x7f6cec0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc02c700 tx=0x7f6cfc032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:00.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.791+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6cf809bf80 con 0x7f6d00103400 2024-03-18T23:09:00.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.964+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d00062420 con 0x7f6d00103400 2024-03-18T23:09:00.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.964+0000 7f6cf77fe640 1 -- 172.21.15.132:0/1359479563 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f6cf809b880 con 0x7f6d00103400 2024-03-18T23:09:00.965 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 msgr2=0x7f6cec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 0x7f6cec0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc02c700 tx=0x7f6cfc032590 comp rx=0 tx=0).stop 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 msgr2=0x7f6d0019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800e970 tx=0x7f6cf800ed70 comp rx=0 tx=0).stop 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 shutdown_connections 2024-03-18T23:09:00.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6cec0a2800 0x7f6cec0a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d00103400 0x7f6d0019f0f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d00102a40 0x7f6d00198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 --2- 172.21.15.132:0/1359479563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d00101850 0x7f6d00197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 >> 172.21.15.132:0/1359479563 conn(0x7f6d000fcdc0 msgr2=0x7f6d00105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 shutdown_connections 2024-03-18T23:09:00.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:00.966+0000 7f6d083c6640 1 -- 172.21.15.132:0/1359479563 wait complete. 2024-03-18T23:09:00.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 -- 172.21.15.132:0/2658530863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 msgr2=0x7fed58108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/2658530863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed58108b80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed50009ca0 tx=0x7fed5002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 -- 172.21.15.132:0/2658530863 shutdown_connections 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/2658530863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed58108b80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/2658530863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58101930 0x7fed58101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/2658530863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58100740 0x7fed58100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.082 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 -- 172.21.15.132:0/2658530863 >> 172.21.15.132:0/2658530863 conn(0x7fed580fbcb0 msgr2=0x7fed580fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 -- 172.21.15.132:0/2658530863 shutdown_connections 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.079+0000 7fed5cdb9640 1 -- 172.21.15.132:0/2658530863 wait complete. 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 Processor -- start 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 -- start start 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 0x7fed58197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58101930 0x7fed58197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810d4e0 con 0x7fed581022f0 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810d360 con 0x7fed58100740 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5cdb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810d660 con 0x7fed58101930 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed56d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed56d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50558/0 (socket says 172.21.15.132:50558) 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed56d9d640 1 -- 172.21.15.132:0/3143897957 learned_addr learned my addr 172.21.15.132:0/3143897957 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.083 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed5659c640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 0x7fed58197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.080+0000 7fed55d9b640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58101930 0x7fed58197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 -- 172.21.15.132:0/3143897957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58101930 msgr2=0x7fed58197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58101930 0x7fed58197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 -- 172.21.15.132:0/3143897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 msgr2=0x7fed58197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 0x7fed58197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 -- 172.21.15.132:0/3143897957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5819f6d0 con 0x7fed581022f0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed5659c640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 0x7fed58197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed56d9d640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed50037040 tx=0x7fed50032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50041020 con 0x7fed581022f0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5819f960 con 0x7fed581022f0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50034cd0 con 0x7fed581022f0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50039040 con 0x7fed581022f0 2024-03-18T23:09:01.084 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.081+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5819fea0 con 0x7fed581022f0 2024-03-18T23:09:01.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.082+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58107a60 con 0x7fed581022f0 2024-03-18T23:09:01.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.083+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fed50044020 con 0x7fed581022f0 2024-03-18T23:09:01.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.084+0000 7fed477fe640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 0x7fed400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.084+0000 7fed5659c640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 0x7fed400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.084+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fed500f08e0 con 0x7fed581022f0 2024-03-18T23:09:01.086 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.084+0000 7fed5659c640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 0x7fed400a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed58101790 tx=0x7fed48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.088+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fed5004b020 con 0x7fed581022f0 2024-03-18T23:09:01.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.302+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed58062420 con 0x7fed581022f0 2024-03-18T23:09:01.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.302+0000 7fed477fe640 1 -- 172.21.15.132:0/3143897957 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11887 (secure 0 0 0) 0x7fed500bda20 con 0x7fed581022f0 2024-03-18T23:09:01.303 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:09:01.303 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:07:18.737695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24493,24518,24485]},"id":1}]} 2024-03-18T23:09:01.304 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 20 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.304+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 msgr2=0x7fed400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.304+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 0x7fed400a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed58101790 tx=0x7fed48005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.304+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 msgr2=0x7fed5819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.304+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed50037040 tx=0x7fed50032f40 comp rx=0 tx=0).stop 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 shutdown_connections 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fed400a2800 0x7fed400a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed581022f0 0x7fed5819f060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.306 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58101930 0x7fed58197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 --2- 172.21.15.132:0/3143897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed58100740 0x7fed58197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 >> 172.21.15.132:0/3143897957 conn(0x7fed580fbcb0 msgr2=0x7fed58104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 shutdown_connections 2024-03-18T23:09:01.307 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.305+0000 7fed5cdb9640 1 -- 172.21.15.132:0/3143897957 wait complete. 2024-03-18T23:09:01.315 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3072692414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.315 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3804525413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.316 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1359479563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.317 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:09:01.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.419+0000 7f6abca61640 1 -- 172.21.15.132:0/2131085782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab8102a20 msgr2=0x7f6ab8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.419+0000 7f6abca61640 1 --2- 172.21.15.132:0/2131085782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab8102a20 0x7f6ab8102ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:01.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- 172.21.15.132:0/2131085782 shutdown_connections 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- 172.21.15.132:0/2131085782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab81033e0 0x7f6ab8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- 172.21.15.132:0/2131085782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab8102a20 0x7f6ab8102ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- 172.21.15.132:0/2131085782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 0x7f6ab8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- 172.21.15.132:0/2131085782 >> 172.21.15.132:0/2131085782 conn(0x7f6ab80fce00 msgr2=0x7f6ab80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- 172.21.15.132:0/2131085782 shutdown_connections 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- 172.21.15.132:0/2131085782 wait complete. 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 Processor -- start 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- start start 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 0x7f6ab80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 0x7f6ab8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810d4b0 con 0x7f6ab81033e0 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810d330 con 0x7f6ab8102a20 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.420+0000 7f6abca61640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810d630 con 0x7f6ab8101830 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 0x7f6ab8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50584/0 (socket says 172.21.15.132:50584) 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab6d9d640 1 -- 172.21.15.132:0/1286953689 learned_addr learned my addr 172.21.15.132:0/1286953689 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab659c640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 0x7f6ab80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 -- 172.21.15.132:0/1286953689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 msgr2=0x7f6ab80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 0x7f6ab80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 -- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 msgr2=0x7f6ab8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 -- 172.21.15.132:0/1286953689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab80749e0 con 0x7f6ab8102a20 2024-03-18T23:09:01.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab6d9d640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6ab5d9b640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 0x7f6ab8073c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009230 tx=0x7f6aa002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0041070 con 0x7f6ab8102a20 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa0004450 con 0x7f6ab8102a20 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab81a1c10 con 0x7f6ab8102a20 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003df0 con 0x7f6ab8102a20 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.421+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81a2160 con 0x7f6ab8102a20 2024-03-18T23:09:01.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.422+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab8108b50 con 0x7f6ab8102a20 2024-03-18T23:09:01.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.423+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6aa0044070 con 0x7f6ab8102a20 2024-03-18T23:09:01.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.424+0000 7f6aaf7fe640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 0x7f6aa40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.424+0000 7f6ab659c640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 0x7f6aa40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.424+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f6aa00f0490 con 0x7f6ab8102a20 2024-03-18T23:09:01.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.424+0000 7f6ab659c640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 0x7f6aa40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8102880 tx=0x7f6ab0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.428 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.428+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6aa00bcef0 con 0x7f6ab8102a20 2024-03-18T23:09:01.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3072692414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3804525413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1359479563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.606+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ab8062420 con 0x7f6ab8102a20 2024-03-18T23:09:01.608 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.607+0000 7f6aaf7fe640 1 -- 172.21.15.132:0/1286953689 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22699 (secure 0 0 0) 0x7f6aa00c0f90 con 0x7f6ab8102a20 2024-03-18T23:09:01.608 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:09:01.608 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":90,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:08:20.188080+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:09:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3072692414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3804525413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1359479563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:09:01.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 msgr2=0x7f6aa40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 0x7f6aa40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8102880 tx=0x7f6ab0009500 comp rx=0 tx=0).stop 2024-03-18T23:09:01.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 msgr2=0x7f6ab8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 0x7f6ab8073c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009230 tx=0x7f6aa002c650 comp rx=0 tx=0).stop 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 shutdown_connections 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f6aa40a2800 0x7f6aa40a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ab81033e0 0x7f6ab8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab8102a20 0x7f6ab8073c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 --2- 172.21.15.132:0/1286953689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ab8101830 0x7f6ab80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 >> 172.21.15.132:0/1286953689 conn(0x7f6ab80fce00 msgr2=0x7f6ab8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 shutdown_connections 2024-03-18T23:09:01.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.609+0000 7f6abca61640 1 -- 172.21.15.132:0/1286953689 wait complete. 2024-03-18T23:09:01.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T23:07:13.853025+0000', 'modified': '2024-03-18T23:07:18.737695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/769602526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 769602526}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 769602526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/559973533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 559973533}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 559973533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6837/3483317240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3483317240}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3483317240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6837/4044803735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 4044803735}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 4044803735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.196:6839/1134284387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 1134284387}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 1134284387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6839/837888145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 837888145}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 837888145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24493, 24518, 24485]} 2024-03-18T23:09:01.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-18T23:09:01.622 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- 172.21.15.132:0/2574563293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 msgr2=0x7f1038101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 --2- 172.21.15.132:0/2574563293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038101d90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1028009900 tx=0x7f102802bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- 172.21.15.132:0/2574563293 shutdown_connections 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 --2- 172.21.15.132:0/2574563293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 0x7f1038108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 --2- 172.21.15.132:0/2574563293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038101d90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 --2- 172.21.15.132:0/2574563293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 0x7f1038100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- 172.21.15.132:0/2574563293 >> 172.21.15.132:0/2574563293 conn(0x7f10380fbcb0 msgr2=0x7f10380fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- 172.21.15.132:0/2574563293 shutdown_connections 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- 172.21.15.132:0/2574563293 wait complete. 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 Processor -- start 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.724+0000 7f10400f8640 1 -- start start 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 0x7f10381046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 0x7f1038103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c3a0 con 0x7f1038101910 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c220 con 0x7f10381022d0 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f103810c520 con 0x7f1038100720 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48874/0 (socket says 172.21.15.132:48874) 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 -- 172.21.15.132:0/2544104343 learned_addr learned my addr 172.21.15.132:0/2544104343 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103de94640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 0x7f10381046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103e695640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 0x7f1038103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 -- 172.21.15.132:0/2544104343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 msgr2=0x7f10381046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 0x7f10381046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 -- 172.21.15.132:0/2544104343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 msgr2=0x7f1038103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 0x7f1038103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 -- 172.21.15.132:0/2544104343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1038103ac0 con 0x7f1038101910 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f103d693640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f102802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f102effd640 1 -- 172.21.15.132:0/2544104343 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028041070 con 0x7f1038101910 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f102effd640 1 -- 172.21.15.132:0/2544104343 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1028004450 con 0x7f1038101910 2024-03-18T23:09:01.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10381a1c40 con 0x7f1038101910 2024-03-18T23:09:01.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f102effd640 1 -- 172.21.15.132:0/2544104343 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028003e20 con 0x7f1038101910 2024-03-18T23:09:01.730 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.725+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10381a2190 con 0x7f1038101910 2024-03-18T23:09:01.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.727+0000 7f102effd640 1 -- 172.21.15.132:0/2544104343 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f1028044070 con 0x7f1038101910 2024-03-18T23:09:01.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 Processor -- start 2024-03-18T23:09:01.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 -- start start 2024-03-18T23:09:01.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13bff0 0x56090b13c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.738 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f102effd640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 0x7f10240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56090b0a0de0 con 0x56090b13c9e0 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56090b049c80 con 0x56090b141b50 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f1042dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56090b09ce80 con 0x56090b13bff0 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48880/0 (socket says 172.21.15.132:48880) 2024-03-18T23:09:01.739 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103de94640 1 -- 172.21.15.132:0/142132483 learned_addr learned my addr 172.21.15.132:0/142132483 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103d693640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 0x7f10240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103e695640 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.728+0000 7f103e695640 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13bff0 0x56090b13c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f102effd640 1 -- 172.21.15.132:0/2544104343 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f10280f0630 con 0x7f1038101910 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 -- 172.21.15.132:0/142132483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13bff0 msgr2=0x56090b13c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13bff0 0x56090b13c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 -- 172.21.15.132:0/142132483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 msgr2=0x56090b143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 -- 172.21.15.132:0/142132483 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090b05bfb0 con 0x56090b13c9e0 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103de94640 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1034009ca0 tx=0x7f1034011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f13107c9cf3d132 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f103d693640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 0x7f10240a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f10280bcfc0 tx=0x7f10280e79d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f1023fff640 1 -- 172.21.15.132:0/142132483 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034004020 con 0x56090b13c9e0 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f1023fff640 1 -- 172.21.15.132:0/142132483 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10340041c0 con 0x56090b13c9e0 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.729+0000 7f1023fff640 1 -- 172.21.15.132:0/142132483 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034019780 con 0x56090b13c9e0 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- 172.21.15.132:0/142132483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 msgr2=0x56090b1414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1034009ca0 tx=0x7f1034011810 comp rx=0 tx=0).stop 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- 172.21.15.132:0/142132483 shutdown_connections 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b1414e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 --2- 172.21.15.132:0/142132483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13bff0 0x56090b13c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- 172.21.15.132:0/142132483 >> 172.21.15.132:0/142132483 conn(0x56090b137560 msgr2=0x56090b137960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- 172.21.15.132:0/142132483 shutdown_connections 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- 172.21.15.132:0/142132483 wait complete. 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 -- start start 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.730+0000 7f1042dc3740 1 Processor -- start 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b14da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56090b144b00 con 0x56090b13c9e0 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56090b05cfc0 con 0x56090b13bff0 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56090b144c80 con 0x56090b141b50 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 10 client.0 _ll_register_callbacks cb 0x56090b098a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37458/0 (socket says 172.21.15.132:37458) 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 -- 172.21.15.132:0/167424591 learned_addr learned my addr 172.21.15.132:0/167424591 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48892/0 (socket says 172.21.15.132:48892) 2024-03-18T23:09:01.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 -- 172.21.15.132:0/167424591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 msgr2=0x56090b14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 -- 172.21.15.132:0/167424591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 msgr2=0x56090b146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 -- 172.21.15.132:0/167424591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090adb52d0 con 0x56090b13bff0 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103de94640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f103e695640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1030012e00 tx=0x7f1030004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8d6c766b6350d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f10217fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030004b20 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f103001a040 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10340041c0 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1030004cc0 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 10 client.25073 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.731+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56090ae7f6b0 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10340041c0 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f103000e750 con 0x56090b13bff0 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 10 client.25073 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:01.745 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 10 client.25073 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 20 client.25073 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1532f0 0x56090b154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 4 client.25073 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 --> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56090b0ae320 con 0x56090b1532f0 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f103de94640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1532f0 0x56090b154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f10217fa640 10 client.25073 ms_handle_connect on v2:172.21.15.144:6836/4044803735 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.732+0000 7f103de94640 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1532f0 0x56090b154760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56559d5d4447b4af server_cookie=da96a232c48e6c73 in_seq=0 out_seq=0 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f10217fa640 1 -- 172.21.15.132:0/167424591 <== mds.2 v2:172.21.15.144:6836/4044803735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56090b0ae320 con 0x56090b1532f0 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f10217fa640 10 client.25073 handle_command_reply: tid=0 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 client.25073 shutdown 2024-03-18T23:09:01.746 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 msgr2=0x56090b146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1030012e00 tx=0x7f1030004a40 comp rx=0 tx=0).stop 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 shutdown_connections 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1532f0 0x56090b154760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 --2- 172.21.15.132:0/167424591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b146570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 >> 172.21.15.132:0/167424591 conn(0x56090b137560 msgr2=0x56090b1367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f10217fa640 0 client.25073 ms_handle_reset on v2:172.21.15.144:6836/4044803735 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 shutdown_connections 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.733+0000 7f1042dc3740 1 -- 172.21.15.132:0/167424591 wait complete. 2024-03-18T23:09:01.747 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.734+0000 7f1042dc3740 20 client.25073 trim_cache size 0 max 16384 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 Processor -- start 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 -- start start 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56090b152aa0 con 0x56090b13c9e0 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56090b153170 con 0x56090b13bff0 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f1042dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56090b153840 con 0x56090b141b50 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.758 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103e695640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103e695640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37466/0 (socket says 172.21.15.132:37466) 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103de94640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48908/0 (socket says 172.21.15.132:48908) 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103e695640 1 -- 172.21.15.132:0/3252827881 learned_addr learned my addr 172.21.15.132:0/3252827881 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103e695640 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.751+0000 7f103e695640 1 -- 172.21.15.132:0/3252827881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 msgr2=0x56090b146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103e695640 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103e695640 1 -- 172.21.15.132:0/3252827881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 msgr2=0x56090b14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103e695640 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103e695640 1 -- 172.21.15.132:0/3252827881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090b13d240 con 0x56090b13bff0 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103de94640 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:01.759 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f103e695640 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f103000f7e0 tx=0x7f1030000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b198beeb7fdc75ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f10237fe640 1 -- 172.21.15.132:0/3252827881 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030030070 con 0x56090b13bff0 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f10237fe640 1 -- 172.21.15.132:0/3252827881 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1030031040 con 0x56090b13bff0 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f10237fe640 1 -- 172.21.15.132:0/3252827881 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030012040 con 0x56090b13bff0 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 -- 172.21.15.132:0/3252827881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 msgr2=0x56090b137940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f103000f7e0 tx=0x7f1030000f40 comp rx=0 tx=0).stop 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 -- 172.21.15.132:0/3252827881 shutdown_connections 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b141b50 0x56090b146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13c9e0 0x56090b14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 --2- 172.21.15.132:0/3252827881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b13bff0 0x56090b137940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.752+0000 7f1042dc3740 1 -- 172.21.15.132:0/3252827881 >> 172.21.15.132:0/3252827881 conn(0x56090b137560 msgr2=0x56090b13d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- 172.21.15.132:0/3252827881 shutdown_connections 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- 172.21.15.132:0/3252827881 wait complete. 2024-03-18T23:09:01.760 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- start start 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 Processor -- start 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 0x56090b14a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56090b1545e0 con 0x56090b13bff0 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56090b153f10 con 0x56090b141b50 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56090b154cb0 con 0x56090b13c9e0 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f1042dc3740 10 client.0 _ll_register_callbacks cb 0x7f10300128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f103e695640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f103e695640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48920/0 (socket says 172.21.15.132:48920) 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f103e695640 1 -- 172.21.15.132:0/1672363521 learned_addr learned my addr 172.21.15.132:0/1672363521 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.761 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f103de94640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 0x56090b14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.753+0000 7f103e695640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 -- 172.21.15.132:0/1672363521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 msgr2=0x56090b14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 0x56090b14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 -- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 msgr2=0x56090b14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 -- 172.21.15.132:0/1672363521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090b14d180 con 0x56090b13bff0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103de94640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 0x56090b14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f103e695640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f103000eb90 tx=0x7f103000ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce11c68729bddbe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030041070 con 0x56090b13bff0 2024-03-18T23:09:01.762 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1030031040 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f103003b9d0 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1030030050 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 10 client.15222 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56090b1b94e0 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1030031040 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f1030046940 con 0x56090b13bff0 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 10 client.15222 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 20 client.15222 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1bdc90 0x56090b1bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.763 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 4 client.15222 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.754+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 --> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] -- command(tid 0: {"prefix": "status"}) -- 0x56090b0a8670 con 0x56090b1bdc90 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f103de94640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1bdc90 0x56090b1bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f10217fa640 10 client.15222 ms_handle_connect on v2:172.21.15.144:6836/4044803735 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f103de94640 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1bdc90 0x56090b1bc9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a51f442cbafa37d server_cookie=99dfc5a77cd6424c in_seq=0 out_seq=0 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f10217fa640 1 -- 172.21.15.132:0/1672363521 <== mds.2 v2:172.21.15.144:6836/4044803735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56090b0a8670 con 0x56090b1bdc90 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f10217fa640 10 client.15222 handle_command_reply: tid=0 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 client.15222 shutdown 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 msgr2=0x56090b149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f103000eb90 tx=0x7f103000ffa0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 shutdown_connections 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:6836/4044803735,v1:172.21.15.144:6837/4044803735] conn(0x56090b1bdc90 0x56090b1bc9a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.764 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56090b141b50 0x56090b14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56090b13c9e0 0x56090b14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 --2- 172.21.15.132:0/1672363521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56090b13bff0 0x56090b149bd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f10217fa640 0 client.15222 ms_handle_reset on v2:172.21.15.144:6836/4044803735 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 >> 172.21.15.132:0/1672363521 conn(0x56090b137560 msgr2=0x56090b137940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 shutdown_connections 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 1 -- 172.21.15.132:0/1672363521 wait complete. 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.755+0000 7f1042dc3740 20 client.15222 trim_cache size 0 max 16384 2024-03-18T23:09:01.765 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 2, 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24485, 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 103.895429017, 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:09:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:09:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 123.67244188799999 2024-03-18T23:09:01.767 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:09:01.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 msgr2=0x7f10240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 0x7f10240a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f10280bcfc0 tx=0x7f10280e79d0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 msgr2=0x7f1038102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1028009230 tx=0x7f102802c650 comp rx=0 tx=0).stop 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 shutdown_connections 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f10240a2800 0x7f10240a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10381022d0 0x7f1038103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1038101910 0x7f1038102cf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 --2- 172.21.15.132:0/2544104343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038100720 0x7f10381046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 >> 172.21.15.132:0/2544104343 conn(0x7f10380fbcb0 msgr2=0x7f10380fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 shutdown_connections 2024-03-18T23:09:01.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.756+0000 7f10400f8640 1 -- 172.21.15.132:0/2544104343 wait complete. 2024-03-18T23:09:01.770 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T23:09:01.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- 172.21.15.132:0/599443207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 msgr2=0x7f75e4100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- 172.21.15.132:0/599443207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e4100e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- 172.21.15.132:0/599443207 shutdown_connections 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- 172.21.15.132:0/599443207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41013d0 0x7f75e4101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- 172.21.15.132:0/599443207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e4100e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- 172.21.15.132:0/599443207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41000d0 0x7f75e41004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- 172.21.15.132:0/599443207 >> 172.21.15.132:0/599443207 conn(0x7f75e40fbcb0 msgr2=0x7f75e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- 172.21.15.132:0/599443207 shutdown_connections 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- 172.21.15.132:0/599443207 wait complete. 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 Processor -- start 2024-03-18T23:09:01.875 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- start start 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41000d0 0x7f75e4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41013d0 0x7f75e41a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410f6b0 con 0x7f75e4100a30 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410f530 con 0x7f75e41013d0 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75e9497640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410f830 con 0x7f75e41000d0 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48938/0 (socket says 172.21.15.132:48938) 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.872+0000 7f75dbfff640 1 -- 172.21.15.132:0/2113949598 learned_addr learned my addr 172.21.15.132:0/2113949598 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.876 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41000d0 0x7f75e4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e37fe640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41013d0 0x7f75e41a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 -- 172.21.15.132:0/2113949598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41000d0 msgr2=0x7f75e4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41000d0 0x7f75e4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 -- 172.21.15.132:0/2113949598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41013d0 msgr2=0x7f75e41a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41013d0 0x7f75e41a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 -- 172.21.15.132:0/2113949598 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e41a19d0 con 0x7f75e4100a30 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75dbfff640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005ab0 tx=0x7f75cc0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e0ff9640 1 -- 172.21.15.132:0/2113949598 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc041070 con 0x7f75e4100a30 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e41a1c60 con 0x7f75e4100a30 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e0ff9640 1 -- 172.21.15.132:0/2113949598 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75cc004450 con 0x7f75e4100a30 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e0ff9640 1 -- 172.21.15.132:0/2113949598 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003e50 con 0x7f75e4100a30 2024-03-18T23:09:01.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.873+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e41a2150 con 0x7f75e4100a30 2024-03-18T23:09:01.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.875+0000 7f75e0ff9640 1 -- 172.21.15.132:0/2113949598 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f75cc044070 con 0x7f75e4100a30 2024-03-18T23:09:01.888 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 Processor -- start 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 -- start start 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e0ff9640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 0x7f75d00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e37fe640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 0x7f75d00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e0ff9640 1 -- 172.21.15.132:0/2113949598 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f75cc0f0590 con 0x7f75e4100a30 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.889 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdc1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5637bcd1ede0 con 0x5637bcdbfb50 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637bccc7c80 con 0x5637bcdba9e0 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75ec082740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637bcd1ae80 con 0x5637bcdb9ff0 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37504/0 (socket says 172.21.15.132:37504) 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:34364/0 (socket says 172.21.15.132:34364) 2024-03-18T23:09:01.890 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 -- 172.21.15.132:0/4035858148 learned_addr learned my addr 172.21.15.132:0/4035858148 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e37fe640 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdc1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 -- 172.21.15.132:0/4035858148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 msgr2=0x5637bcdba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 -- 172.21.15.132:0/4035858148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 msgr2=0x5637bcdc1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75e37fe640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 0x7f75d00a4e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fd0 tx=0x7f75dc011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdc1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.876+0000 7f75dbfff640 1 -- 172.21.15.132:0/4035858148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637bccd9fb0 con 0x5637bcdba9e0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.877+0000 7f75dbfff640 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0bcdb0 tx=0x7f75cc0bcbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01ec65d2869afba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.877+0000 7f75d97fa640 1 -- 172.21.15.132:0/4035858148 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc04b040 con 0x5637bcdba9e0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.877+0000 7f75d97fa640 1 -- 172.21.15.132:0/4035858148 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75cc0c1150 con 0x5637bcdba9e0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.877+0000 7f75d97fa640 1 -- 172.21.15.132:0/4035858148 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0b9720 con 0x5637bcdba9e0 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- 172.21.15.132:0/4035858148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 msgr2=0x5637bcdbf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0bcdb0 tx=0x7f75cc0bcbf0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.891 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- 172.21.15.132:0/4035858148 shutdown_connections 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdc1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdbf4e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- 172.21.15.132:0/4035858148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- 172.21.15.132:0/4035858148 >> 172.21.15.132:0/4035858148 conn(0x5637bcdb5560 msgr2=0x5637bcdb5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- 172.21.15.132:0/4035858148 shutdown_connections 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- 172.21.15.132:0/4035858148 wait complete. 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- start start 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 Processor -- start 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdc4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdc28c0 con 0x5637bcdbfb50 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637bccdafc0 con 0x5637bcdba9e0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdc2a40 con 0x5637bcdb9ff0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75ec082740 10 client.0 _ll_register_callbacks cb 0x5637bcd16a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48964/0 (socket says 172.21.15.132:48964) 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75e37fe640 1 -- 172.21.15.132:0/4217127121 learned_addr learned my addr 172.21.15.132:0/4217127121 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdc4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.878+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 -- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 msgr2=0x5637bcdc41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 -- 172.21.15.132:0/4217127121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 msgr2=0x5637bcdc4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdc4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 -- 172.21.15.132:0/4217127121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637bca332d0 con 0x5637bcdbfb50 2024-03-18T23:09:01.893 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75e37fe640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01db30 tx=0x7f75dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123dcbc6e52410e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc017040 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f75dc009040 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75cc0c1150 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75dc00f880 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 10 client.15234 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5637bcafd6b0 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75cc0c1150 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f75dc0323f0 con 0x5637bcdbfb50 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 10 client.15234 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 20 client.15234 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bcdd10f0 0x5637bcdd2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 4 client.15234 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.879+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 --> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637bcd2c320 con 0x5637bcdd10f0 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.880+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bcdd10f0 0x5637bcdd2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.880+0000 7f75ca7fc640 10 client.15234 ms_handle_connect on v2:172.21.15.196:6838/1134284387 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.880+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bcdd10f0 0x5637bcdd2560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e76f18e6bff88a8 server_cookie=496c0225fe5362e5 in_seq=0 out_seq=0 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ca7fc640 1 -- 172.21.15.132:0/4217127121 <== mds.1 v2:172.21.15.196:6838/1134284387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637bcd2c320 con 0x5637bcdd10f0 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ca7fc640 10 client.15234 handle_command_reply: tid=0 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 client.15234 shutdown 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 msgr2=0x5637bcdcb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01db30 tx=0x7f75dc004ab0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 shutdown_connections 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bcdd10f0 0x5637bcdd2560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.895 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcb760 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdc4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ca7fc640 0 client.15234 ms_handle_reset on v2:172.21.15.196:6838/1134284387 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 --2- 172.21.15.132:0/4217127121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 >> 172.21.15.132:0/4217127121 conn(0x5637bcdb5560 msgr2=0x5637bcdb47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 shutdown_connections 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 1 -- 172.21.15.132:0/4217127121 wait complete. 2024-03-18T23:09:01.896 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.881+0000 7f75ec082740 20 client.15234 trim_cache size 0 max 16384 2024-03-18T23:09:01.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.898+0000 7f75ec082740 1 Processor -- start 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 -- start start 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdbbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdcb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd08a0 con 0x5637bcdbfb50 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd0f70 con 0x5637bcdba9e0 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75ec082740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd1640 con 0x5637bcdb9ff0 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48970/0 (socket says 172.21.15.132:48970) 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 -- 172.21.15.132:0/3627012545 learned_addr learned my addr 172.21.15.132:0/3627012545 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75dbfff640 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdcb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdbbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 -- 172.21.15.132:0/3627012545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 msgr2=0x5637bcdbbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdbbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 -- 172.21.15.132:0/3627012545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 msgr2=0x5637bcdcb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdcb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 -- 172.21.15.132:0/3627012545 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637bcdb0fc0 con 0x5637bcdbfb50 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01d570 tx=0x7f75dc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6051dc533b69d409 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.899+0000 7f75d8ff9640 1 -- 172.21.15.132:0/3627012545 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc017070 con 0x5637bcdbfb50 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75d8ff9640 1 -- 172.21.15.132:0/3627012545 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75dc019050 con 0x5637bcdbfb50 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75d8ff9640 1 -- 172.21.15.132:0/3627012545 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00fa20 con 0x5637bcdbfb50 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- 172.21.15.132:0/3627012545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 msgr2=0x5637bcdbc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01d570 tx=0x7f75dc005f10 comp rx=0 tx=0).stop 2024-03-18T23:09:01.906 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- 172.21.15.132:0/3627012545 shutdown_connections 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdbc420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdba9e0 0x5637bcdcb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 --2- 172.21.15.132:0/3627012545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdb9ff0 0x5637bcdbbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- 172.21.15.132:0/3627012545 >> 172.21.15.132:0/3627012545 conn(0x5637bcdb5560 msgr2=0x5637bcdc2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- 172.21.15.132:0/3627012545 shutdown_connections 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- 172.21.15.132:0/3627012545 wait complete. 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.900+0000 7f75ec082740 1 -- start start 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 Processor -- start 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdba9e0 0x5637bcdc7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd23e0 con 0x5637bcdbfb50 2024-03-18T23:09:01.907 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd1d10 con 0x5637bcdb9ff0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5637bcdd2ab0 con 0x5637bcdba9e0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 10 client.0 _ll_register_callbacks cb 0x7f75dc01d5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48978/0 (socket says 172.21.15.132:48978) 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 -- 172.21.15.132:0/3538475742 learned_addr learned my addr 172.21.15.132:0/3538475742 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75dbfff640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdba9e0 0x5637bcdc7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 -- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdba9e0 msgr2=0x5637bcdc7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdba9e0 0x5637bcdc7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 -- 172.21.15.132:0/3538475742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdb9ff0 msgr2=0x5637bcdc7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 -- 172.21.15.132:0/3538475742 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637bcdcb040 con 0x5637bcdbfb50 2024-03-18T23:09:01.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75e37fe640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fa0 tx=0x7f75dc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead02764645aa76 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc03e070 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f75dc00fc30 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.901+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75dc019050 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75dc04a050 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 10 client.15240 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5637bce37510 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75dc019050 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f75dc005510 con 0x5637bcdbfb50 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 10 client.15240 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 20 client.15240 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:01.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bce3bc30 0x5637bce3a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 4 client.15240 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 --> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] -- command(tid 0: {"prefix": "status"}) -- 0x5637bcd26670 con 0x5637bce3bc30 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bce3bc30 0x5637bce3a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75ca7fc640 10 client.15240 ms_handle_connect on v2:172.21.15.196:6838/1134284387 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.902+0000 7f75e2ffd640 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bce3bc30 0x5637bce3a940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80606c0e73300ac server_cookie=d01311106741c631 in_seq=0 out_seq=0 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ca7fc640 1 -- 172.21.15.132:0/3538475742 <== mds.1 v2:172.21.15.196:6838/1134284387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5637bcd26670 con 0x5637bce3bc30 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ca7fc640 10 client.15240 handle_command_reply: tid=0 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 client.15240 shutdown 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 msgr2=0x5637bcdcaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fa0 tx=0x7f75dc009650 comp rx=0 tx=0).stop 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 shutdown_connections 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:6838/1134284387,v1:172.21.15.196:6839/1134284387] conn(0x5637bce3bc30 0x5637bce3a940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637bcdbfb50 0x5637bcdcaa00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5637bcdba9e0 0x5637bcdc7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 --2- 172.21.15.132:0/3538475742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637bcdb9ff0 0x5637bcdc7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ca7fc640 0 client.15240 ms_handle_reset on v2:172.21.15.196:6838/1134284387 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 >> 172.21.15.132:0/3538475742 conn(0x5637bcdb5560 msgr2=0x5637bcdc2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 shutdown_connections 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 1 -- 172.21.15.132:0/3538475742 wait complete. 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75ec082740 20 client.15240 trim_cache size 0 max 16384 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:09:01.911 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 1, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24493, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 105.04816153100001, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 121.026742274 2024-03-18T23:09:01.912 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:09:01.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 msgr2=0x7f75d00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 0x7f75d00a4e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fd0 tx=0x7f75dc011810 comp rx=0 tx=0).stop 2024-03-18T23:09:01.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 msgr2=0x7f75e419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005ab0 tx=0x7f75cc0049c0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 shutdown_connections 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f75d00a29a0 0x7f75d00a4e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e41013d0 0x7f75e41a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e4100a30 0x7f75e419a2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 --2- 172.21.15.132:0/2113949598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75e41000d0 0x7f75e4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 >> 172.21.15.132:0/2113949598 conn(0x7f75e40fbcb0 msgr2=0x7f75e40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.903+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 shutdown_connections 2024-03-18T23:09:01.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:01.904+0000 7f75e9497640 1 -- 172.21.15.132:0/2113949598 wait complete. 2024-03-18T23:09:01.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T23:09:02.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 -- 172.21.15.132:0/1273019901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c8101830 msgr2=0x7fb1c8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 --2- 172.21.15.132:0/1273019901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c8101830 0x7fb1c8101c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009860 tx=0x7fb1b002bb70 comp rx=0 tx=0).stop 2024-03-18T23:09:02.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 -- 172.21.15.132:0/1273019901 shutdown_connections 2024-03-18T23:09:02.020 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 --2- 172.21.15.132:0/1273019901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c81033e0 0x7fb1c8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 --2- 172.21.15.132:0/1273019901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 --2- 172.21.15.132:0/1273019901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c8101830 0x7fb1c8101c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 -- 172.21.15.132:0/1273019901 >> 172.21.15.132:0/1273019901 conn(0x7fb1c80fcdc0 msgr2=0x7fb1c80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 -- 172.21.15.132:0/1273019901 shutdown_connections 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.017+0000 7fb1cde44640 1 -- 172.21.15.132:0/1273019901 wait complete. 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 Processor -- start 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 -- start start 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 0x7fb1c8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 0x7fb1c819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810d480 con 0x7fb1c8102a20 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810d300 con 0x7fb1c8101830 2024-03-18T23:09:02.021 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1cde44640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c810d600 con 0x7fb1c81033e0 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49006/0 (socket says 172.21.15.132:49006) 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/261379840 learned_addr learned my addr 172.21.15.132:0/261379840 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 0x7fb1c819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.018+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 0x7fb1c8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/261379840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 msgr2=0x7fb1c819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 0x7fb1c819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/261379840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 msgr2=0x7fb1c8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 0x7fb1c8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/261379840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c819f790 con 0x7fb1c8102a20 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 0x7fb1c819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 0x7fb1c8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800e9c0 tx=0x7fb1b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c4ff9640 1 -- 172.21.15.132:0/261379840 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800cde0 con 0x7fb1c8102a20 2024-03-18T23:09:02.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c819fa80 con 0x7fb1c8102a20 2024-03-18T23:09:02.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c4ff9640 1 -- 172.21.15.132:0/261379840 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b80035f0 con 0x7fb1c8102a20 2024-03-18T23:09:02.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1c4ff9640 1 -- 172.21.15.132:0/261379840 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8010650 con 0x7fb1c8102a20 2024-03-18T23:09:02.023 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.019+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c819ffd0 con 0x7fb1c8102a20 2024-03-18T23:09:02.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.021+0000 7fb1c4ff9640 1 -- 172.21.15.132:0/261379840 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fb1b80037a0 con 0x7fb1c8102a20 2024-03-18T23:09:02.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.021+0000 7fb1d0a2b740 1 Processor -- start 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 -- start start 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c4ff9640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 0x7fb1b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b5b50 0x55b2b31b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3114de0 con 0x55b2b31b09e0 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2b30bdc80 con 0x55b2b31afff0 2024-03-18T23:09:02.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b3110e80 con 0x55b2b31b5b50 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 0x7fb1b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37552/0 (socket says 172.21.15.132:37552) 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 -- 172.21.15.132:0/1063130527 learned_addr learned my addr 172.21.15.132:0/1063130527 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:02.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b5b50 0x55b2b31b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c4ff9640 1 -- 172.21.15.132:0/261379840 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb1b80ced30 con 0x7fb1c8102a20 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 -- 172.21.15.132:0/1063130527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b5b50 msgr2=0x55b2b31b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b5b50 0x55b2b31b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 -- 172.21.15.132:0/1063130527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 msgr2=0x55b2b31b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 -- 172.21.15.132:0/1063130527 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b30cffb0 con 0x55b2b31afff0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009860 tx=0x7fb1b0034ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9377b456e4c960 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 0x7fb1b40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00099e0 tx=0x7fb1c0009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.022+0000 7fb1bdffb640 1 -- 172.21.15.132:0/1063130527 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b004b070 con 0x55b2b31afff0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1bdffb640 1 -- 172.21.15.132:0/1063130527 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0032340 con 0x55b2b31afff0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1bdffb640 1 -- 172.21.15.132:0/1063130527 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0039040 con 0x55b2b31afff0 2024-03-18T23:09:02.035 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/1063130527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 msgr2=0x55b2b31b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009860 tx=0x7fb1b0034ad0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/1063130527 shutdown_connections 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b5b50 0x55b2b31b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/1063130527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b0410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/1063130527 >> 172.21.15.132:0/1063130527 conn(0x55b2b31ab560 msgr2=0x55b2b31ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/1063130527 shutdown_connections 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/1063130527 wait complete. 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.023+0000 7fb1d0a2b740 1 -- start start 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 Processor -- start 2024-03-18T23:09:02.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 0x55b2b31c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31b8aa0 con 0x55b2b31b09e0 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2b30d0fc0 con 0x55b2b31b5b50 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31b8c20 con 0x55b2b31afff0 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1d0a2b740 10 client.0 _ll_register_callbacks cb 0x55b2b310ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49024/0 (socket says 172.21.15.132:49024) 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/2362174504 learned_addr learned my addr 172.21.15.132:0/2362174504 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:02.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 0x55b2b31c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/2362174504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 msgr2=0x55b2b31ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 msgr2=0x55b2b31c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 0x55b2b31c19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 -- 172.21.15.132:0/2362174504 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b2e292d0 con 0x55b2b31b09e0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 0x55b2b31c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8051010 tx=0x7fb1b809b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d536c12ad7459 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.024+0000 7fb1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8066b90 con 0x55b2b31b09e0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb1b80999e0 con 0x55b2b31b09e0 2024-03-18T23:09:02.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b0032340 con 0x55b2b31b09e0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1b809b6e0 con 0x55b2b31b09e0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 10 client.15252 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b2b2ef36b0 con 0x55b2b31b09e0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0032340 con 0x55b2b31b09e0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb1b809ab30 con 0x55b2b31b09e0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 10 client.15252 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 20 client.15252 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b31c7260 0x55b2b31c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 4 client.15252 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b3122320 con 0x55b2b31c7260 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.025+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b31c7260 0x55b2b31c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b31c7260 0x55b2b31c86d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989ea6c3cd34e8c0 server_cookie=6a950f6695f99cb3 in_seq=0 out_seq=0 2024-03-18T23:09:02.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1a6ffd640 10 client.15252 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2362174504 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b2b3122320 con 0x55b2b31c7260 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1a6ffd640 10 client.15252 handle_command_reply: tid=0 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 client.15252 shutdown 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 msgr2=0x55b2b31ba990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8051010 tx=0x7fb1b809b5c0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 shutdown_connections 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b31c7260 0x55b2b31c86d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.026+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b5b50 0x55b2b31c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b09e0 0x55b2b31ba990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2362174504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 >> 172.21.15.132:0/2362174504 conn(0x55b2b31ab560 msgr2=0x55b2b31aa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1a6ffd640 0 client.15252 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 shutdown_connections 2024-03-18T23:09:02.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2362174504 wait complete. 2024-03-18T23:09:02.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.027+0000 7fb1d0a2b740 20 client.15252 trim_cache size 0 max 16384 2024-03-18T23:09:02.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 Processor -- start 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 -- start start 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b09e0 0x55b2b31c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c6a10 con 0x55b2b31b5b50 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c70e0 con 0x55b2b31afff0 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c77b0 con 0x55b2b31b09e0 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.044+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49030/0 (socket says 172.21.15.132:49030) 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 -- 172.21.15.132:0/3478407231 learned_addr learned my addr 172.21.15.132:0/3478407231 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b09e0 0x55b2b31c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 -- 172.21.15.132:0/3478407231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b09e0 msgr2=0x55b2b31c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b09e0 0x55b2b31c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 -- 172.21.15.132:0/3478407231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 msgr2=0x55b2b31b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 -- 172.21.15.132:0/3478407231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b31bb210 con 0x55b2b31b5b50 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005e30 tx=0x7fb1c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13313cb1be4af6cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1bd7fa640 1 -- 172.21.15.132:0/3478407231 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0013170 con 0x55b2b31b5b50 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1bd7fa640 1 -- 172.21.15.132:0/3478407231 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0003c80 con 0x55b2b31b5b50 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.045+0000 7fb1bd7fa640 1 -- 172.21.15.132:0/3478407231 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0010690 con 0x55b2b31b5b50 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/3478407231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 msgr2=0x55b2b31ba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0005e30 tx=0x7fb1c0004ab0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/3478407231 shutdown_connections 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31ba450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31b09e0 0x55b2b31c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/3478407231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31afff0 0x55b2b31b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/3478407231 >> 172.21.15.132:0/3478407231 conn(0x55b2b31ab560 msgr2=0x55b2b31c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/3478407231 shutdown_connections 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/3478407231 wait complete. 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- start start 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 Processor -- start 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 0x55b2b31bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c8550 con 0x55b2b31b5b50 2024-03-18T23:09:02.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c7e80 con 0x55b2b31b09e0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b2b31c8c20 con 0x55b2b31afff0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1d0a2b740 10 client.0 _ll_register_callbacks cb 0x7fb1b80972c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:34444/0 (socket says 172.21.15.132:34444) 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.046+0000 7fb1c77fe640 1 -- 172.21.15.132:0/2515151183 learned_addr learned my addr 172.21.15.132:0/2515151183 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 0x55b2b31bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 -- 172.21.15.132:0/2515151183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 msgr2=0x55b2b31bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 -- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 msgr2=0x55b2b31bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 0x55b2b31bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 -- 172.21.15.132:0/2515151183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b31c0750 con 0x55b2b31b5b50 2024-03-18T23:09:02.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c6ffd640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 0x55b2b31bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1c7fff640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31c0160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004a80 tx=0x7fb1c0013920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a38d6b70ad83b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004b20 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb1c0004cc0 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c0003c80 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1c0023430 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1d0a2b740 10 client.15258 fetch_fsmap learned FSMap version 20 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b2b31c1a50 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0003c80 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.047+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fb1c0034c40 con 0x55b2b31b5b50 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 10 client.15258 fetch_fsmap finished waiting for FSMap version 20 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T23:09:02.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 20 client.15258 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b3231310 0x55b2b3230020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 4 client.15258 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "status"}) -- 0x55b2b311c670 con 0x55b2b3231310 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b3231310 0x55b2b3230020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1a6ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1c77fe640 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b3231310 0x55b2b3230020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2f544b51d35db7 server_cookie=b34c5893e5681be3 in_seq=0 out_seq=0 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1a6ffd640 1 -- 172.21.15.132:0/2515151183 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b2b311c670 con 0x55b2b3231310 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1a6ffd640 10 client.15258 handle_command_reply: tid=0 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 client.15258 shutdown 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 msgr2=0x55b2b31c0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31c0160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004a80 tx=0x7fb1c0013920 comp rx=0 tx=0).stop 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 shutdown_connections 2024-03-18T23:09:02.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55b2b3231310 0x55b2b3230020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2b31b5b50 0x55b2b31c0160 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2b31b09e0 0x55b2b31bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1a6ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 --2- 172.21.15.132:0/2515151183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b2b31afff0 0x55b2b31bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.048+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 >> 172.21.15.132:0/2515151183 conn(0x55b2b31ab560 msgr2=0x55b2b31b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 shutdown_connections 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1d0a2b740 1 -- 172.21.15.132:0/2515151183 wait complete. 2024-03-18T23:09:02.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1d0a2b740 20 client.15258 trim_cache size 0 max 16384 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "cluster_fsid": "0a41095a-e57b-11ee-95c9-87774f69a715", 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "whoami": 0, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24518, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "want_state": "up:active", 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "state": "up:active", 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs", 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "rank_uptime": 107.21291415500001, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "mdsmap_epoch": 20, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch": 90, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_epoch_barrier": 88, 2024-03-18T23:09:02.058 INFO:teuthology.orchestra.run.smithi132.stdout: "uptime": 112.95442278100001 2024-03-18T23:09:02.059 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:09:02.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 msgr2=0x7fb1b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 0x7fb1b40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00099e0 tx=0x7fb1c0009f00 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 msgr2=0x7fb1c81980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800e9c0 tx=0x7fb1b800ee80 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 shutdown_connections 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7fb1b40a2800 0x7fb1b40a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1c81033e0 0x7fb1c819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1c8102a20 0x7fb1c81980b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 --2- 172.21.15.132:0/261379840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c8101830 0x7fb1c8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 >> 172.21.15.132:0/261379840 conn(0x7fb1c80fcdc0 msgr2=0x7fb1c8105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 shutdown_connections 2024-03-18T23:09:02.060 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:02.049+0000 7fb1cde44640 1 -- 172.21.15.132:0/261379840 wait complete. 2024-03-18T23:09:02.061 INFO:teuthology.run_tasks:Running task kclient... 2024-03-18T23:09:02.088 INFO:tasks.kclient:Mounting kernel clients... 2024-03-18T23:09:02.088 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-18T23:09:02.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:02.088 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-18T23:09:02.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:02.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-18T23:09:02.172 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-03-18T23:09:02.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:09:02.174 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-18T23:09:02.174 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T23:09:02.175 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:02.175 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T23:09:02.451 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:09:02.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:02 smithi196 ceph-mon[67089]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:09:02.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:02 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3143897957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:02.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:02 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1286953689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:02.573 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:02 smithi132 ceph-mon[32020]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:09:02.573 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:02 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3143897957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:02.573 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:02 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1286953689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:02 smithi144 ceph-mon[66781]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-18T23:09:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:02 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3143897957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:09:02.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:02 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1286953689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:09:04.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/4126362377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102360 msgr2=0x7f0ed01027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:04.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/4126362377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102360 0x7f0ed01027e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009990 tx=0x7f0ecc009ca0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/4126362377 shutdown_connections 2024-03-18T23:09:04.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/4126362377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 0x7f0ed01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/4126362377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102d20 0x7f0ed010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/4126362377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102360 0x7f0ed01027e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/4126362377 >> 172.21.15.132:0/4126362377 conn(0x7f0ed0079440 msgr2=0x7f0ed0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/4126362377 shutdown_connections 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.144+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/4126362377 wait complete. 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 Processor -- start 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 -- start start 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102d20 0x7f0ed019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 0x7f0ed01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0112620 con 0x7f0ed0102d20 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01124a0 con 0x7f0ed0108410 2024-03-18T23:09:04.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.145+0000 7f0ed7f1a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed01127a0 con 0x7f0ed0102360 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:34460/0 (socket says 172.21.15.132:34460) 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 -- 172.21.15.132:0/2742042443 learned_addr learned my addr 172.21.15.132:0/2742042443 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed54b5700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102d20 0x7f0ed019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed64b7700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 0x7f0ed01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 -- 172.21.15.132:0/2742042443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 msgr2=0x7f0ed01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 0x7f0ed01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 -- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102d20 msgr2=0x7f0ed019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102d20 0x7f0ed019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 -- 172.21.15.132:0/2742042443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0199000 con 0x7f0ed0102360 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed5cb6700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000e9a0 tx=0x7f0ec000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000ca10 con 0x7f0ed0102360 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec000cbb0 con 0x7f0ed0102360 2024-03-18T23:09:04.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed01992f0 con 0x7f0ed0102360 2024-03-18T23:09:04.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0018680 con 0x7f0ed0102360 2024-03-18T23:09:04.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.146+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed01a55b0 con 0x7f0ed0102360 2024-03-18T23:09:04.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.147+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed0068fd0 con 0x7f0ed0102360 2024-03-18T23:09:04.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.148+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f0ec0047a60 con 0x7f0ed0102360 2024-03-18T23:09:04.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.149+0000 7f0ec6ffd700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 0x7f0ebc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:04.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.149+0000 7f0ed54b5700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 0x7f0ebc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:04.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.149+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0ec00ceca0 con 0x7f0ed0102360 2024-03-18T23:09:04.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.149+0000 7f0ed54b5700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 0x7f0ebc0a4d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0098e0 tx=0x7f0ecc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:04.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.152+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ec009bd50 con 0x7f0ed0102360 2024-03-18T23:09:04.395 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.394+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0ed002dbb0 con 0x7f0ed0102360 2024-03-18T23:09:04.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.395+0000 7f0ec6ffd700 1 -- 172.21.15.132:0/2742042443 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f0ec009b650 con 0x7f0ed0102360 2024-03-18T23:09:04.396 INFO:teuthology.orchestra.run.smithi132.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T23:09:04.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 msgr2=0x7f0ebc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:04.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 0x7f0ebc0a4d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0098e0 tx=0x7f0ecc005960 comp rx=0 tx=0).stop 2024-03-18T23:09:04.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 msgr2=0x7f0ed019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:04.398 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000e9a0 tx=0x7f0ec000ecb0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 shutdown_connections 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f0ebc0a2890 0x7f0ebc0a4d50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed0108410 0x7f0ed01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ed0102360 0x7f0ed019e6e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 --2- 172.21.15.132:0/2742042443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed0102d20 0x7f0ed019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 >> 172.21.15.132:0/2742042443 conn(0x7f0ed0079440 msgr2=0x7f0ed0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 shutdown_connections 2024-03-18T23:09:04.399 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:04.397+0000 7f0ed7f1a700 1 -- 172.21.15.132:0/2742042443 wait complete. 2024-03-18T23:09:04.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:04 smithi196 ceph-mon[67089]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-18T23:09:04.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:04 smithi144 ceph-mon[66781]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-18T23:09:04.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:04 smithi132 ceph-mon[32020]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T23:09:05.088 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T23:09:05.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:05.088 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-03-18T23:09:05.105 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T23:09:05.105 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T23:09:05.105 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T23:09:05.105 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:09:05.105 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-03-18T23:09:05.106 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-18T23:09:05.107 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 40454sec preferred_lft 40454sec 2024-03-18T23:09:05.107 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-03-18T23:09:05.107 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-03-18T23:09:05.107 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T23:09:05.107 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:09:05.107 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-18T23:09:05.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-03-18T23:09:05.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-03-18T23:09:05.107 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-03-18T23:09:05.108 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T23:09:05.108 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-18T23:09:05.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:05.305 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:05.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:05.311 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T23:09:05.318 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:05 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2742042443' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:09:05.338 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-03-18T23:09:05.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:05.340 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-03-18T23:09:05.416 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-03-18T23:09:05.417 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T23:09:05.417 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-18T23:09:05.417 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-18T23:09:05.417 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T23:09:05.417 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:09:05.417 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-18T23:09:05.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-18T23:09:05.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-18T23:09:05.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-18T23:09:05.418 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-18T23:09:05.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:05 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2742042443' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:09:05.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:05 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2742042443' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T23:09:05.742 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:05.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:05.749 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-18T23:09:05.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:05.767 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-03-18T23:09:05.822 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:09:05.822 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-18T23:09:05.822 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T23:09:05.822 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T23:09:05.822 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-18T23:09:05.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:05.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:05.945 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T23:09:05.945 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:09:05.945 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-18T23:09:05.945 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T23:09:05.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T23:09:05.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T23:09:05.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T23:09:05.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T23:09:05.946 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-18T23:09:05.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:06.079 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.086 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T23:09:06.086 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-03-18T23:09:06.086 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-03-18T23:09:06.086 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T23:09:06.086 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-03-18T23:09:06.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:06.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T23:09:06.168 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:06.183 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T23:09:06.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T23:09:06.183 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:06.238 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-18T23:09:06.238 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-18T23:09:06.238 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/e3a84061-a703-4239-8f60-e71c218ee9e0 2024-03-18T23:09:06.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:06.238 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/e3a84061-a703-4239-8f60-e71c218ee9e0 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-18T23:09:06.322 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-18T23:09:06.323 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-03-18T23:09:06.323 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-03-18T23:09:06.434 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-18T23:09:06.434 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-03-18T23:09:06.434 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-03-18T23:09:06.434 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.132:3300,172.21.15.144:3300,172.21.15.196:3300" 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.132:3300,172.21.15.144:3300,172.21.15.196:3300:/volumes/qa/sv_1/e3a84061-a703-4239-8f60-e71c218ee9e0 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=0a41095a-e57b-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-18T23:09:06.435 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-03-18T23:09:06.436 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-18T23:09:06.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T23:09:06.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:09:06.437 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:06.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:06 smithi196 ceph-mon[67089]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:06.501 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:06 smithi132 ceph-mon[32020]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:06.509 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:09:06.598 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:09:06.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.606 INFO:teuthology.orchestra.run.smithi132.stdout:client25124 2024-03-18T23:09:06.606 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:09:06.606 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:09:06.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:06 smithi144 ceph-mon[66781]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:06.623 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:09:06.711 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:09:06.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.718 INFO:teuthology.orchestra.run.smithi132.stdout:client25124 2024-03-18T23:09:06.719 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T23:09:06.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T23:09:06.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.774 INFO:teuthology.orchestra.run.smithi132.stdout:client25124 2024-03-18T23:09:06.775 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0a41095a-e57b-11ee-95c9-87774f69a715.client25124/status') 2024-03-18T23:09:06.823 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T23:09:06.835 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.25124 (3)172.21.15.132:0/1539184643 2024-03-18T23:09:06.835 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-03-18T23:09:06.835 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-03-18T23:09:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-03-18T23:09:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:72 bytes copied, 2.1426e-05 s, 3.4 MB/s 2024-03-18T23:09:06.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T23:09:06.843 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T23:09:06.852 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:09:06.853 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:09:06.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- 172.21.15.132:0/134289414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf81013f0 msgr2=0x7f1cf8108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/134289414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf81013f0 0x7f1cf8108c90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf002bef0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- 172.21.15.132:0/134289414 shutdown_connections 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/134289414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf81013f0 0x7f1cf8108c90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/134289414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 0x7f1cf8100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/134289414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf8101d00 0x7f1cf8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- 172.21.15.132:0/134289414 >> 172.21.15.132:0/134289414 conn(0x7f1cf80fb480 msgr2=0x7f1cf80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:06.958 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- 172.21.15.132:0/134289414 shutdown_connections 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- 172.21.15.132:0/134289414 wait complete. 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 Processor -- start 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.955+0000 7f1cfe92f640 1 -- start start 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 0x7f1cf8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf81013f0 0x7f1cf8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810d520 con 0x7f1cf8101d00 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810d3a0 con 0x7f1cf81013f0 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810d6a0 con 0x7f1cf8100a30 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cf77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf81013f0 0x7f1cf8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49064/0 (socket says 172.21.15.132:49064) 2024-03-18T23:09:06.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 -- 172.21.15.132:0/2908503963 learned_addr learned my addr 172.21.15.132:0/2908503963 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cf7fff640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 0x7f1cf8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 -- 172.21.15.132:0/2908503963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 msgr2=0x7f1cf8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 0x7f1cf8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 -- 172.21.15.132:0/2908503963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf81013f0 msgr2=0x7f1cf8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf81013f0 0x7f1cf8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 -- 172.21.15.132:0/2908503963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf819f680 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfcecc640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0036040 tx=0x7f1cf0033d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0043020 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf002ca60 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0034720 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.956+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf819f910 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.957+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf819feb0 con 0x7f1cf8101d00 2024-03-18T23:09:06.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.957+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf8107aa0 con 0x7f1cf8101d00 2024-03-18T23:09:06.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.959+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f1cf0046020 con 0x7f1cf8101d00 2024-03-18T23:09:06.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.959+0000 7f1cf57fa640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 0x7f1ce00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:06.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.959+0000 7f1cf7fff640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 0x7f1ce00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:06.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.959+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f1cf00f18a0 con 0x7f1cf8101d00 2024-03-18T23:09:06.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.960+0000 7f1cf7fff640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 0x7f1ce00a4de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8069de0 tx=0x7f1ce8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:06.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:06.963+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1cf004d020 con 0x7f1cf8101d00 2024-03-18T23:09:07.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.177+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f1cf802d1f0 con 0x7f1cf8101d00 2024-03-18T23:09:07.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.209+0000 7f1cf57fa640 1 -- 172.21.15.132:0/2908503963 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v21) ==== 117+0+0 (secure 0 0 0) 0x7f1cf00be950 con 0x7f1cf8101d00 2024-03-18T23:09:07.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.211+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 msgr2=0x7f1ce00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:07.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.211+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 0x7f1ce00a4de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8069de0 tx=0x7f1ce8005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:07.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.211+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 msgr2=0x7f1cf819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.211+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0036040 tx=0x7f1cf0033d80 comp rx=0 tx=0).stop 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 shutdown_connections 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f1ce00a2920 0x7f1ce00a4de0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8101d00 0x7f1cf819f010 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf81013f0 0x7f1cf8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 --2- 172.21.15.132:0/2908503963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cf8100a30 0x7f1cf8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 >> 172.21.15.132:0/2908503963 conn(0x7f1cf80fb480 msgr2=0x7f1cf8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 shutdown_connections 2024-03-18T23:09:07.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.212+0000 7f1cfe92f640 1 -- 172.21.15.132:0/2908503963 wait complete. 2024-03-18T23:09:07.223 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-18T23:09:07.234 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T23:09:07.268 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:09:07.268 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:09:07.269 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-18T23:09:07.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- 172.21.15.132:0/56087835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c100740 msgr2=0x7f062c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:07.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- 172.21.15.132:0/56087835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c100740 0x7f062c100b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0628009860 tx=0x7f062802bb70 comp rx=0 tx=0).stop 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- 172.21.15.132:0/56087835 shutdown_connections 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- 172.21.15.132:0/56087835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c1022f0 0x7f062c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- 172.21.15.132:0/56087835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c101930 0x7f062c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- 172.21.15.132:0/56087835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c100740 0x7f062c100b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- 172.21.15.132:0/56087835 >> 172.21.15.132:0/56087835 conn(0x7f062c0fbcb0 msgr2=0x7f062c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- 172.21.15.132:0/56087835 shutdown_connections 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- 172.21.15.132:0/56087835 wait complete. 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 Processor -- start 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 -- start start 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.370+0000 7f0634085640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c101930 0x7f062c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 0x7f062c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:07.373 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f062c10c3b0 con 0x7f062c100740 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f062c10c230 con 0x7f062c1022f0 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f062c10c530 con 0x7f062c101930 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49096/0 (socket says 172.21.15.132:49096) 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 -- 172.21.15.132:0/1843336090 learned_addr learned my addr 172.21.15.132:0/1843336090 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631620640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c101930 0x7f062c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0632622640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 0x7f062c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 -- 172.21.15.132:0/1843336090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c101930 msgr2=0x7f062c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c101930 0x7f062c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 -- 172.21.15.132:0/1843336090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 msgr2=0x7f062c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 0x7f062c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 -- 172.21.15.132:0/1843336090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c19e670 con 0x7f062c100740 2024-03-18T23:09:07.374 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0632622640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 0x7f062c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0631e21640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f062802c0b0 tx=0x7f0628032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628041020 con 0x7f062c100740 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062802c9f0 con 0x7f062c100740 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c19e900 con 0x7f062c100740 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628039040 con 0x7f062c100740 2024-03-18T23:09:07.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.371+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c19edf0 con 0x7f062c100740 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.372+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c107ab0 con 0x7f062c100740 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.373+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f0628044020 con 0x7f062c100740 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.374+0000 7f0622ffd640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 0x7f06180a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.374+0000 7f0631620640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 0x7f06180a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.374+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f06280f03f0 con 0x7f062c100740 2024-03-18T23:09:07.376 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.374+0000 7f0631620640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 0x7f06180a4d60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f062c197f60 tx=0x7f061c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:07.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.378+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f062804b020 con 0x7f062c100740 2024-03-18T23:09:07.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:07 smithi196 ceph-mon[67089]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:09:07.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:07 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T23:09:07.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:07.600+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f062c062420 con 0x7f062c100740 2024-03-18T23:09:07.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:07 smithi144 ceph-mon[66781]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:09:07.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:07 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T23:09:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:07 smithi132 ceph-mon[32020]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-18T23:09:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:07 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T23:09:08.218 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.218+0000 7f0622ffd640 1 -- 172.21.15.132:0/1843336090 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f06280bd4a0 con 0x7f062c100740 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 msgr2=0x7f06180a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 0x7f06180a4d60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f062c197f60 tx=0x7f061c005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 msgr2=0x7f062c196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f062802c0b0 tx=0x7f0628032430 comp rx=0 tx=0).stop 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 shutdown_connections 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f06180a28a0 0x7f06180a4d60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062c1022f0 0x7f062c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.221 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062c101930 0x7f062c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 --2- 172.21.15.132:0/1843336090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f062c100740 0x7f062c196a50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 >> 172.21.15.132:0/1843336090 conn(0x7f062c0fbcb0 msgr2=0x7f062c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.220+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 shutdown_connections 2024-03-18T23:09:08.222 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.221+0000 7f0634085640 1 -- 172.21.15.132:0/1843336090 wait complete. 2024-03-18T23:09:08.240 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-18T23:09:08.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 -- 172.21.15.132:0/4089775714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c101930 msgr2=0x7f482c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 --2- 172.21.15.132:0/4089775714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c101930 0x7f482c101db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4820009900 tx=0x7f482002bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:08.353 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 -- 172.21.15.132:0/4089775714 shutdown_connections 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 --2- 172.21.15.132:0/4089775714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 --2- 172.21.15.132:0/4089775714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c101930 0x7f482c101db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 --2- 172.21.15.132:0/4089775714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c100740 0x7f482c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 -- 172.21.15.132:0/4089775714 >> 172.21.15.132:0/4089775714 conn(0x7f482c0fbcb0 msgr2=0x7f482c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 -- 172.21.15.132:0/4089775714 shutdown_connections 2024-03-18T23:09:08.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.352+0000 7f483207e640 1 -- 172.21.15.132:0/4089775714 wait complete. 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.353+0000 7f483207e640 1 Processor -- start 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.353+0000 7f483207e640 1 -- start start 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 0x7f482c196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10c3c0 con 0x7f482c100740 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10c240 con 0x7f482c101930 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10c540 con 0x7f482c1022f0 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49130/0 (socket says 172.21.15.132:49130) 2024-03-18T23:09:08.357 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 -- 172.21.15.132:0/1751436338 learned_addr learned my addr 172.21.15.132:0/1751436338 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482bfff640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482affd640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 0x7f482c196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 -- 172.21.15.132:0/1751436338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 msgr2=0x7f482c19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 -- 172.21.15.132:0/1751436338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 msgr2=0x7f482c196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 0x7f482c196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 -- 172.21.15.132:0/1751436338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19e5c0 con 0x7f482c100740 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482bfff640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482affd640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 0x7f482c196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f482b7fe640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9c0 tx=0x7f481c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c00cd80 con 0x7f482c100740 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c0035f0 con 0x7f482c100740 2024-03-18T23:09:08.358 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c010690 con 0x7f482c100740 2024-03-18T23:09:08.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.354+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19e8b0 con 0x7f482c100740 2024-03-18T23:09:08.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.355+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c19edd0 con 0x7f482c100740 2024-03-18T23:09:08.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.356+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c107a60 con 0x7f482c100740 2024-03-18T23:09:08.359 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.357+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) ==== 137308+0+0 (secure 0 0 0) 0x7f481c0037a0 con 0x7f482c100740 2024-03-18T23:09:08.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.357+0000 7f4828ff9640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 0x7f48140a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.357+0000 7f482affd640 1 -- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 msgr2=0x7f48140a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:08.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.357+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f481c0cee60 con 0x7f482c100740 2024-03-18T23:09:08.360 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.357+0000 7f482affd640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 0x7f48140a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:09:08.362 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.362+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f481c09bf10 con 0x7f482c100740 2024-03-18T23:09:08.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T23:09:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:09:08.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:09:08.474 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: ignoring --setuser ceph since I am not root 2024-03-18T23:09:08.474 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: ignoring --setgroup ceph since I am not root 2024-03-18T23:09:08.474 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-mgr[69027]: -- 172.21.15.196:0/2264526737 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2f7a365a0 con 0x55a2f7a0ac00 2024-03-18T23:09:08.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.543+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f482c062450 con 0x7f482c100740 2024-03-18T23:09:08.547 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.547+0000 7f4828ff9640 1 -- 172.21.15.132:0/1751436338 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f481c09b810 con 0x7f482c100740 2024-03-18T23:09:08.549 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 msgr2=0x7f48140a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 0x7f48140a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 msgr2=0x7f482c1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9c0 tx=0x7f481c00ee80 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 shutdown_connections 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f48140a28f0 0x7f48140a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f482c1022f0 0x7f482c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101930 0x7f482c196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 --2- 172.21.15.132:0/1751436338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f482c100740 0x7f482c1969a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 >> 172.21.15.132:0/1751436338 conn(0x7f482c0fbcb0 msgr2=0x7f482c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 shutdown_connections 2024-03-18T23:09:08.550 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.549+0000 7f483207e640 1 -- 172.21.15.132:0/1751436338 wait complete. 2024-03-18T23:09:08.560 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T23:09:08.560 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:09:08.560 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:09:08.561 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setuser ceph since I am not root 2024-03-18T23:09:08.561 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: ignoring --setgroup ceph since I am not root 2024-03-18T23:09:08.562 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2908503963' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: ignoring --setuser ceph since I am not root 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: ignoring --setgroup ceph since I am not root 2024-03-18T23:09:08.609 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-mgr[68162]: -- 172.21.15.144:0/3530392528 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac719365a0 con 0x55ac7190a800 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 -- 172.21.15.132:0/1847557094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d41022f0 msgr2=0x7f39d4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1847557094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d41022f0 0x7f39d4108b80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009ca0 tx=0x7f39cc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 -- 172.21.15.132:0/1847557094 shutdown_connections 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1847557094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d41022f0 0x7f39d4108b80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1847557094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4101930 0x7f39d4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1847557094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4100740 0x7f39d4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 -- 172.21.15.132:0/1847557094 >> 172.21.15.132:0/1847557094 conn(0x7f39d40fbcb0 msgr2=0x7f39d40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.666 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 -- 172.21.15.132:0/1847557094 shutdown_connections 2024-03-18T23:09:08.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.665+0000 7f39d9aea640 1 -- 172.21.15.132:0/1847557094 wait complete. 2024-03-18T23:09:08.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 Processor -- start 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 -- start start 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4100740 0x7f39d41046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d41022f0 0x7f39d4103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410c3c0 con 0x7f39d4101930 2024-03-18T23:09:08.669 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410c240 con 0x7f39d41022f0 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d9aea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39d410c540 con 0x7f39d4100740 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49156/0 (socket says 172.21.15.132:49156) 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 -- 172.21.15.132:0/1578469260 learned_addr learned my addr 172.21.15.132:0/1578469260 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d3fff640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d41022f0 0x7f39d4103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d37fe640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4100740 0x7f39d41046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 -- 172.21.15.132:0/1578469260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4100740 msgr2=0x7f39d41046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4100740 0x7f39d41046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 -- 172.21.15.132:0/1578469260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d41022f0 msgr2=0x7f39d4103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d41022f0 0x7f39d4103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.666+0000 7f39d2ffd640 1 -- 172.21.15.132:0/1578469260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4103ae0 con 0x7f39d4101930 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d2ffd640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e9f0 tx=0x7f39c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:08.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800cd80 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c8005500 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d41a1be0 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8010690 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.667+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41a20e0 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.668+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d41a2310 con 0x7f39d4101930 2024-03-18T23:09:08.671 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.669+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) ==== 137308+0+0 (secure 0 0 0) 0x7f39c80046c0 con 0x7f39d4101930 2024-03-18T23:09:08.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.669+0000 7f39d0ff9640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 0x7f39bc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.670+0000 7f39d37fe640 1 -- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 msgr2=0x7f39bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:08.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.670+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f39c80cef30 con 0x7f39d4101930 2024-03-18T23:09:08.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.670+0000 7f39d37fe640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 0x7f39bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:09:08.674 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.674+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f39c809bfe0 con 0x7f39d4101930 2024-03-18T23:09:08.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.854+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f39d41a24e0 con 0x7f39d4101930 2024-03-18T23:09:08.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.858+0000 7f39d0ff9640 1 -- 172.21.15.132:0/1578469260 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f39c8084e20 con 0x7f39d4101930 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 msgr2=0x7f39bc0a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 0x7f39bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 msgr2=0x7f39d4102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e9f0 tx=0x7f39c800edf0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 shutdown_connections 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f39bc0a28f0 0x7f39bc0a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d41022f0 0x7f39d4103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d4101930 0x7f39d4102d10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 --2- 172.21.15.132:0/1578469260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39d4100740 0x7f39d41046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 >> 172.21.15.132:0/1578469260 conn(0x7f39d40fbcb0 msgr2=0x7f39d40fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 shutdown_connections 2024-03-18T23:09:08.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.861+0000 7f39d9aea640 1 -- 172.21.15.132:0/1578469260 wait complete. 2024-03-18T23:09:08.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 -- 172.21.15.132:0/1600141610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0102a60 msgr2=0x7f52d0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 --2- 172.21.15.132:0/1600141610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0102a60 0x7f52d0102ee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009900 tx=0x7f52b802bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 -- 172.21.15.132:0/1600141610 shutdown_connections 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 --2- 172.21.15.132:0/1600141610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d0103420 0x7f52d0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 --2- 172.21.15.132:0/1600141610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0102a60 0x7f52d0102ee0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 --2- 172.21.15.132:0/1600141610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0101870 0x7f52d0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 -- 172.21.15.132:0/1600141610 >> 172.21.15.132:0/1600141610 conn(0x7f52d00fce00 msgr2=0x7f52d00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:08.981 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 -- 172.21.15.132:0/1600141610 shutdown_connections 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.978+0000 7f52d5532640 1 -- 172.21.15.132:0/1600141610 wait complete. 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 Processor -- start 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 -- start start 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d0101870 0x7f52d0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 0x7f52d0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0103420 0x7f52d019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d4f0 con 0x7f52d0103420 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d370 con 0x7f52d0101870 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52d5532640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d670 con 0x7f52d0102a60 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 0x7f52d0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52cf7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0103420 0x7f52d019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 0x7f52d0197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:34562/0 (socket says 172.21.15.132:34562) 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 -- 172.21.15.132:0/484635478 learned_addr learned my addr 172.21.15.132:0/484635478 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d0101870 0x7f52d0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d0101870 msgr2=0x7f52d0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d0101870 0x7f52d0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0103420 msgr2=0x7f52d019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0103420 0x7f52d019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52c7fff640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d019f6a0 con 0x7f52d0102a60 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.979+0000 7f52cf7fe640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d0103420 0x7f52d019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52c7fff640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 0x7f52d0197fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009230 tx=0x7f52b802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8041070 con 0x7f52d0102a60 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d019f930 con 0x7f52d0102a60 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b8004450 con 0x7f52d0102a60 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8003e20 con 0x7f52d0102a60 2024-03-18T23:09:08.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.980+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d019fe20 con 0x7f52d0102a60 2024-03-18T23:09:08.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.981+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d0108ac0 con 0x7f52d0102a60 2024-03-18T23:09:08.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.982+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 137308+0+0 (secure 0 0 0) 0x7f52b8044070 con 0x7f52d0102a60 2024-03-18T23:09:08.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.982+0000 7f52ccff9640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:08.984 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.982+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:08.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.982+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f52b80f0910 con 0x7f52d0102a60 2024-03-18T23:09:08.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.982+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T23:09:08.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:08.986+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f52b80bd370 con 0x7f52d0102a60 2024-03-18T23:09:09.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:08.838+0000 7fac6d171200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:09:09.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:08.945+0000 7fac6d171200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:09:09.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:08.818+0000 7fb4d7e08200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:09:09.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:08 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:08.917+0000 7fb4d7e08200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:09:09.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:09.184+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:09.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:09.184+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T23:09:09.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:09.188+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f52d0061c60 con 0x7f52bc0a28f0 2024-03-18T23:09:09.220 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:08.790+0000 7fbd58561200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T23:09:09.220 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:08.886+0000 7fbd58561200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T23:09:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:09:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-mon[67089]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-03-18T23:09:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1751436338' entity='client.admin' 2024-03-18T23:09:09.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1578469260' entity='client.admin' 2024-03-18T23:09:09.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:09:09.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-mon[32020]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-03-18T23:09:09.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1751436338' entity='client.admin' 2024-03-18T23:09:09.568 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1578469260' entity='client.admin' 2024-03-18T23:09:09.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:09.585+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:09.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:09.585+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T23:09:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1843336090' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T23:09:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-mon[66781]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-03-18T23:09:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1751436338' entity='client.admin' 2024-03-18T23:09:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1578469260' entity='client.admin' 2024-03-18T23:09:09.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:09.567+0000 7fb4d7e08200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:09:09.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:09.661+0000 7fb4d7e08200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:09:09.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:09.805+0000 7fb4d7e08200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:09:09.879 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:09.539+0000 7fbd58561200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:09:09.880 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:09.633+0000 7fbd58561200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:09:09.880 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:09.778+0000 7fbd58561200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:09:09.880 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:09.879+0000 7fbd58561200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:09:10.055 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:09.718+0000 7fac6d171200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T23:09:10.055 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:09.812+0000 7fac6d171200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T23:09:10.056 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:09.958+0000 7fac6d171200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T23:09:10.056 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.054+0000 7fac6d171200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:09:10.224 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:09.990+0000 7fbd58561200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:09:10.252 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:09.904+0000 7fb4d7e08200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T23:09:10.252 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:09 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:09.987+0000 7fb4d7e08200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:09:10.252 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:10 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:10.251+0000 7fb4d7e08200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:09:10.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.165+0000 7fac6d171200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T23:09:10.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:10.386+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:10.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:10.386+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T23:09:10.543 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:10 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:10.361+0000 7fb4d7e08200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:10.543 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:10 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:10.446+0000 7fb4d7e08200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:09:10.543 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:10 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:10.542+0000 7fb4d7e08200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:09:10.570 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:10.275+0000 7fbd58561200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:09:10.570 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:10.386+0000 7fbd58561200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:10.571 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:10.473+0000 7fbd58561200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:09:10.767 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.470+0000 7fac6d171200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T23:09:10.767 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.582+0000 7fac6d171200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:10.767 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.669+0000 7fac6d171200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T23:09:10.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:10 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:10.719+0000 7fb4d7e08200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:09:10.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:10.570+0000 7fbd58561200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:09:10.974 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:10.751+0000 7fbd58561200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:09:11.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.766+0000 7fac6d171200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T23:09:11.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:10.971+0000 7fac6d171200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T23:09:11.851 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:11 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:11.714+0000 7fb4d7e08200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:09:11.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:11.987+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:11.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:11.987+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T23:09:12.115 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:11 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:11.850+0000 7fb4d7e08200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:09:12.223 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:11.781+0000 7fbd58561200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:09:12.223 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:11.928+0000 7fbd58561200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:09:12.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:12.069+0000 7fac6d171200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T23:09:12.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:12.203+0000 7fac6d171200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T23:09:12.488 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:12.195+0000 7fb4d7e08200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:09:12.488 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:12.287+0000 7fb4d7e08200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:09:12.637 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:12.321+0000 7fbd58561200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:09:12.637 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:12.436+0000 7fbd58561200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:09:12.637 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:12.636+0000 7fbd58561200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:12.858 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:12.567+0000 7fac6d171200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T23:09:12.858 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:12.657+0000 7fac6d171200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T23:09:12.858 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:12.857+0000 7fac6d171200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:12.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:12.492+0000 7fb4d7e08200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:12.865 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:12.681+0000 7fb4d7e08200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:09:12.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:12.823+0000 7fbd58561200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:09:13.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:13 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:13.053+0000 7fac6d171200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T23:09:13.234 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:12 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:12.968+0000 7fb4d7e08200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:13.383 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:13.112+0000 7fbd58561200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:13.609 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:13 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:13.342+0000 7fac6d171200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T23:09:13.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:13.229+0000 7fb4d7e08200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:13.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:13.471+0000 7fb4d7e08200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:09:13.660 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:13.380+0000 7fbd58561200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:13.962 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:13 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:13.640+0000 7fac6d171200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T23:09:13.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:13.659+0000 7fbd58561200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:09:14.359 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:13 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:13.960+0000 7fac6d171200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T23:09:14.615 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:14.294+0000 7fb4d7e08200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:09:14.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:14.540+0000 7fbd58561200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:09:15.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:15.191+0000 7f52ceffd640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.132:6800/3730787943 2024-03-18T23:09:15.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:15.191+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-18T23:09:15.200 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:14.872+0000 7fb4d7e08200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:09:15.200 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:14 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:14.975+0000 7fb4d7e08200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:09:15.200 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.199+0000 7fb4d7e08200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:09:15.224 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:15.224+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mgrmap(e 20) ==== 136975+0+0 (secure 0 0 0) 0x7f52b80035f0 con 0x7f52d0102a60 2024-03-18T23:09:15.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:15.224+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 msgr2=0x7f52bc0a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T23:09:15.225 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:15.224+0000 7f52ccff9640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/3730787943,v1:172.21.15.132:6801/3730787943] conn(0x7f52bc0a28f0 0x7f52bc0a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:15.266 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:15.008+0000 7fac6d171200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T23:09:15.470 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: Active manager daemon x restarted 2024-03-18T23:09:15.470 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: Activating manager daemon x 2024-03-18T23:09:15.470 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:09:15.470 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: mgrmap e20: x(active, starting, since 0.0122049s), standbys: y, z 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:09:15.471 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:09:15.472 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:09:15.473 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:09:15.473 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:09:15.473 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:09:15.473 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:09:15.473 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-mon[32020]: Manager daemon x is now available 2024-03-18T23:09:15.597 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:15.239+0000 7fbd58561200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:15.358+0000 7fbd58561200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: Active manager daemon x restarted 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: Activating manager daemon x 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: mgrmap e20: x(active, starting, since 0.0122049s), standbys: y, z 2024-03-18T23:09:15.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:09:15.599 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:09:15.600 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:09:15.601 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-mon[67089]: Manager daemon x is now available 2024-03-18T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: Active manager daemon x restarted 2024-03-18T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: Activating manager daemon x 2024-03-18T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: mgrmap e20: x(active, starting, since 0.0122049s), standbys: y, z 2024-03-18T23:09:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:09:15.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T23:09:15.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T23:09:15.612 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-mon[66781]: Manager daemon x is now available 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.468+0000 7fb437797700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.468+0000 7fb437797700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.468+0000 7fb437797700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.468+0000 7fb437797700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.468+0000 7fb437797700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.469+0000 7fb43a79d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.469+0000 7fb43a79d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.469+0000 7fb43a79d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.469+0000 7fb43a79d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.866 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:09:15 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:09:15.469+0000 7fb43a79d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:09:15.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:09:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z[68988]: 2024-03-18T23:09:15.596+0000 7fbd58561200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:09:16.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:15.722+0000 7fac6d171200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T23:09:16.109 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:15 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:15.851+0000 7fac6d171200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T23:09:16.231 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.231+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mon.1 v2:172.21.15.196:3300/0 8 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f52b80b99e0 con 0x7f52d0102a60 2024-03-18T23:09:16.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.231+0000 7f52ccff9640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52bc0511d0 0x7f52bc0515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.232 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.231+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f52b800b0b0 con 0x7f52bc0511d0 2024-03-18T23:09:16.236 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.235+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52bc0511d0 0x7f52bc0515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.238 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.238+0000 7f52ceffd640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52bc0511d0 0x7f52bc0515b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f52d01028c0 tx=0x7f52c0005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:16.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.308+0000 7f52ccff9640 1 -- 172.21.15.132:0/484635478 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f52b800b0b0 con 0x7f52bc0511d0 2024-03-18T23:09:16.309 INFO:teuthology.orchestra.run.smithi132.stdout:Schedule set for path / 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52bc0511d0 msgr2=0x7f52bc0515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52bc0511d0 0x7f52bc0515b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f52d01028c0 tx=0x7f52c0005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 msgr2=0x7f52d0197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 --2- 172.21.15.132:0/484635478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52d0102a60 0x7f52d0197fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009230 tx=0x7f52b802c650 comp rx=0 tx=0).stop 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52cf7fe640 1 -- 172.21.15.132:0/484635478 reap_dead start 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 shutdown_connections 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 >> 172.21.15.132:0/484635478 conn(0x7f52d00fce00 msgr2=0x7f52d0105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 shutdown_connections 2024-03-18T23:09:16.311 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.310+0000 7f52d5532640 1 -- 172.21.15.132:0/484635478 wait complete. 2024-03-18T23:09:16.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 -- 172.21.15.132:0/1264019880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb738100740 msgr2=0x7fb738100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/1264019880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb738100740 0x7fb738100b60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009900 tx=0x7fb72c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 -- 172.21.15.132:0/1264019880 shutdown_connections 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/1264019880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7381022f0 0x7fb738108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/1264019880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 0x7fb738101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/1264019880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb738100740 0x7fb738100b60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 -- 172.21.15.132:0/1264019880 >> 172.21.15.132:0/1264019880 conn(0x7fb7380fbcb0 msgr2=0x7fb7380fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:16.432 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 -- 172.21.15.132:0/1264019880 shutdown_connections 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.429+0000 7fb73ca6c640 1 -- 172.21.15.132:0/1264019880 wait complete. 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 Processor -- start 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 -- start start 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb738100740 0x7fb738069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 0x7fb73819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb73810c3b0 con 0x7fb7381022f0 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb73810c230 con 0x7fb738100740 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73ca6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb73810c530 con 0x7fb738101930 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb736d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb736d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42644/0 (socket says 172.21.15.132:42644) 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb736d9d640 1 -- 172.21.15.132:0/2383390586 learned_addr learned my addr 172.21.15.132:0/2383390586 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb73659c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb738100740 0x7fb738069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.433 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.430+0000 7fb735d9b640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 0x7fb73819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 -- 172.21.15.132:0/2383390586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 msgr2=0x7fb73819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 0x7fb73819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 -- 172.21.15.132:0/2383390586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb738100740 msgr2=0x7fb738069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb738100740 0x7fb738069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 -- 172.21.15.132:0/2383390586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7381a19c0 con 0x7fb7381022f0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb736d9d640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb730009fd0 tx=0x7fb73000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730017070 con 0x7fb7381022f0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7381a1c50 con 0x7fb7381022f0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb730003e50 con 0x7fb7381022f0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7300106c0 con 0x7fb7381022f0 2024-03-18T23:09:16.434 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.431+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a21a0 con 0x7fb7381022f0 2024-03-18T23:09:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.432+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb738107ae0 con 0x7fb7381022f0 2024-03-18T23:09:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.433+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7fb730003970 con 0x7fb7381022f0 2024-03-18T23:09:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.433+0000 7fb7277fe640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 0x7fb7200a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.434+0000 7fb73659c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 0x7fb7200a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.435 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.434+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb7300ce420 con 0x7fb7381022f0 2024-03-18T23:09:16.436 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.434+0000 7fb73659c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 0x7fb7200a4ba0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb72c000c00 tx=0x7fb72c005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:16.438 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.438+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb73009b070 con 0x7fb7381022f0 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: Standby manager daemon z restarted 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: Standby manager daemon z started 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: Standby manager daemon y restarted 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: Standby manager daemon y started 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.584 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:16 smithi196 ceph-mon[67089]: mgrmap e21: x(active, since 1.01848s), standbys: z, y 2024-03-18T23:09:16.588 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y[68158]: 2024-03-18T23:09:16.114+0000 7fac6d171200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T23:09:16.588 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: Standby manager daemon z restarted 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: Standby manager daemon z started 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: Standby manager daemon y restarted 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: Standby manager daemon y started 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:09:16.589 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:09:16.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.590 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:16 smithi144 ceph-mon[66781]: mgrmap e21: x(active, since 1.01848s), standbys: z, y 2024-03-18T23:09:16.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: Standby manager daemon z restarted 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: Standby manager daemon z started 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.196:0/2984597513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: Standby manager daemon y restarted 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: Standby manager daemon y started 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T23:09:16.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T23:09:16.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: from='mgr.? 172.21.15.144:0/2706469418' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T23:09:16.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:16 smithi132 ceph-mon[32020]: mgrmap e21: x(active, since 1.01848s), standbys: z, y 2024-03-18T23:09:16.639 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.639+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb738061dc0 con 0x7fb7200a26e0 2024-03-18T23:09:16.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.694+0000 7fb7277fe640 1 -- 172.21.15.132:0/2383390586 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb738061dc0 con 0x7fb7200a26e0 2024-03-18T23:09:16.695 INFO:teuthology.orchestra.run.smithi132.stdout:Retention added to path / 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 msgr2=0x7fb7200a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 0x7fb7200a4ba0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb72c000c00 tx=0x7fb72c005fd0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 msgr2=0x7fb7381a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb730009fd0 tx=0x7fb73000b260 comp rx=0 tx=0).stop 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 shutdown_connections 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb7200a26e0 0x7fb7200a4ba0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7381022f0 0x7fb7381a11d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738101930 0x7fb73819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 --2- 172.21.15.132:0/2383390586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb738100740 0x7fb738069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 >> 172.21.15.132:0/2383390586 conn(0x7fb7380fbcb0 msgr2=0x7fb738104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:16.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 shutdown_connections 2024-03-18T23:09:16.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.696+0000 7fb73ca6c640 1 -- 172.21.15.132:0/2383390586 wait complete. 2024-03-18T23:09:16.708 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T23:09:16.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 -- 172.21.15.132:0/3910516247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0fff10 msgr2=0x7f4e7c100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 --2- 172.21.15.132:0/3910516247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c100330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e7802bc30 comp rx=0 tx=0).stop 2024-03-18T23:09:16.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 -- 172.21.15.132:0/3910516247 shutdown_connections 2024-03-18T23:09:16.815 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 --2- 172.21.15.132:0/3910516247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101a20 0x7f4e7c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 --2- 172.21.15.132:0/3910516247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c101060 0x7f4e7c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 --2- 172.21.15.132:0/3910516247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c100330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.814+0000 7f4e83397640 1 -- 172.21.15.132:0/3910516247 >> 172.21.15.132:0/3910516247 conn(0x7f4e7c0fb480 msgr2=0x7f4e7c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:16.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- 172.21.15.132:0/3910516247 shutdown_connections 2024-03-18T23:09:16.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- 172.21.15.132:0/3910516247 wait complete. 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 Processor -- start 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- start start 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 0x7f4e7c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10d570 con 0x7f4e7c101a20 2024-03-18T23:09:16.820 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10d3f0 con 0x7f4e7c101060 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.815+0000 7f4e83397640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10d6f0 con 0x7f4e7c0fff10 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81133640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42660/0 (socket says 172.21.15.132:42660) 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 -- 172.21.15.132:0/3642311092 learned_addr learned my addr 172.21.15.132:0/3642311092 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e80932640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 0x7f4e7c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 -- 172.21.15.132:0/3642311092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c0fff10 msgr2=0x7f4e7c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 -- 172.21.15.132:0/3642311092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 msgr2=0x7f4e7c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 0x7f4e7c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 -- 172.21.15.132:0/3642311092 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19f6b0 con 0x7f4e7c101a20 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e80932640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 0x7f4e7c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e81934640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400eb10 tx=0x7f4e7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:16.821 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74004b20 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e74004cc0 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19f9a0 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74010760 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.816+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c19fea0 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.817+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c1071c0 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.818+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7f4e74005020 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.819+0000 7f4e727fc640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 0x7f4e680a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.819+0000 7f4e81133640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 0x7f4e680a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.819+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4e740cecc0 con 0x7f4e7c101a20 2024-03-18T23:09:16.822 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.819+0000 7f4e81133640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 0x7f4e680a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e7802c7b0 tx=0x7f4e78005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:16.824 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:16.823+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4e7409bf60 con 0x7f4e7c101a20 2024-03-18T23:09:17.022 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.021+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f4e7c061c60 con 0x7f4e680a26b0 2024-03-18T23:09:17.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.029+0000 7f4e727fc640 1 -- 172.21.15.132:0/3642311092 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f4e7c061c60 con 0x7f4e680a26b0 2024-03-18T23:09:17.029 INFO:teuthology.orchestra.run.smithi132.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T23:09:16", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-18T23:09:17.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 msgr2=0x7f4e680a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 0x7f4e680a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e7802c7b0 tx=0x7f4e78005fd0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 msgr2=0x7f4e7c19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400eb10 tx=0x7f4e7400c2d0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 shutdown_connections 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4e680a26b0 0x7f4e680a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c101a20 0x7f4e7c19f040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c101060 0x7f4e7c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 --2- 172.21.15.132:0/3642311092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4e7c0fff10 0x7f4e7c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 >> 172.21.15.132:0/3642311092 conn(0x7f4e7c0fb480 msgr2=0x7f4e7c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 shutdown_connections 2024-03-18T23:09:17.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.031+0000 7f4e83397640 1 -- 172.21.15.132:0/3642311092 wait complete. 2024-03-18T23:09:17.044 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T23:09:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 -- 172.21.15.132:0/4013429311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384102330 msgr2=0x7fa384108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 --2- 172.21.15.132:0/4013429311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384102330 0x7fa384108bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009ca0 tx=0x7fa37c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 -- 172.21.15.132:0/4013429311 shutdown_connections 2024-03-18T23:09:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 --2- 172.21.15.132:0/4013429311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384102330 0x7fa384108bc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 --2- 172.21.15.132:0/4013429311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 --2- 172.21.15.132:0/4013429311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384100780 0x7fa384100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 -- 172.21.15.132:0/4013429311 >> 172.21.15.132:0/4013429311 conn(0x7fa3840fbcf0 msgr2=0x7fa3840fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 -- 172.21.15.132:0/4013429311 shutdown_connections 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.155+0000 7fa38b48a640 1 -- 172.21.15.132:0/4013429311 wait complete. 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 Processor -- start 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 -- start start 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 0x7fa38419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa38410c3e0 con 0x7fa384100780 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa38410c260 con 0x7fa384102330 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa38b48a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa38410c560 con 0x7fa384101970 2024-03-18T23:09:17.159 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa389226640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa389226640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42676/0 (socket says 172.21.15.132:42676) 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa388a25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa389226640 1 -- 172.21.15.132:0/3484773097 learned_addr learned my addr 172.21.15.132:0/3484773097 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.156+0000 7fa389a27640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 0x7fa38419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 -- 172.21.15.132:0/3484773097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 msgr2=0x7fa384197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 -- 172.21.15.132:0/3484773097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 msgr2=0x7fa38419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 0x7fa38419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 -- 172.21.15.132:0/3484773097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38419f6d0 con 0x7fa384100780 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389a27640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 0x7fa38419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa388a25640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa389226640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9c0 tx=0x7fa38000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38000cd80 con 0x7fa384100780 2024-03-18T23:09:17.160 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3800035f0 con 0x7fa384100780 2024-03-18T23:09:17.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38419f9c0 con 0x7fa384100780 2024-03-18T23:09:17.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380010690 con 0x7fa384100780 2024-03-18T23:09:17.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.157+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38419ff60 con 0x7fa384100780 2024-03-18T23:09:17.161 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.158+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3841a01f0 con 0x7fa384100780 2024-03-18T23:09:17.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.159+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 21) ==== 137102+0+0 (secure 0 0 0) 0x7fa3800037a0 con 0x7fa384100780 2024-03-18T23:09:17.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.160+0000 7fa37a7fc640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 0x7fa3700a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:17.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.160+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa3800cea80 con 0x7fa384100780 2024-03-18T23:09:17.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.160+0000 7fa388a25640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 0x7fa3700a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:17.162 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.160+0000 7fa388a25640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 0x7fa3700a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa384198fc0 tx=0x7fa374005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:17.165 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.164+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa38009bd20 con 0x7fa384100780 2024-03-18T23:09:17.365 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:17 smithi132 ceph-mon[32020]: [18/Mar/2024:23:09:16] ENGINE Bus STARTING 2024-03-18T23:09:17.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.418+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa3841a03c0 con 0x7fa3700a2730 2024-03-18T23:09:17.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.425+0000 7fa37a7fc640 1 -- 172.21.15.132:0/3484773097 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa3841a03c0 con 0x7fa3700a2730 2024-03-18T23:09:17.425 INFO:teuthology.orchestra.run.smithi132.stdout:/ 1m 6m3h 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 msgr2=0x7fa3700a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 0x7fa3700a4bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa384198fc0 tx=0x7fa374005f90 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 msgr2=0x7fa384197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9c0 tx=0x7fa38000ee80 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 shutdown_connections 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa3700a2730 0x7fa3700a4bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa384102330 0x7fa38419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa384101970 0x7fa384197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 --2- 172.21.15.132:0/3484773097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa384100780 0x7fa384197ab0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 >> 172.21.15.132:0/3484773097 conn(0x7fa3840fbcf0 msgr2=0x7fa3840fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 shutdown_connections 2024-03-18T23:09:17.431 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:09:17.428+0000 7fa38b48a640 1 -- 172.21.15.132:0/3484773097 wait complete. 2024-03-18T23:09:17.443 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-18T23:09:17.473 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-18T23:09:17.485 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-18T23:09:17.506 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:09:17.506 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-18T23:09:17.516 INFO:teuthology.run_tasks:Running task workunit... 2024-03-18T23:09:17.537 INFO:tasks.workunit:Pulling workunits from ref e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-18T23:09:17.537 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-18T23:09:17.537 INFO:tasks.workunit:timeout=3h 2024-03-18T23:09:17.537 INFO:tasks.workunit:cleanup=True 2024-03-18T23:09:17.537 DEBUG:teuthology.orchestra.run.smithi132:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:17.552 INFO:teuthology.orchestra.run.smithi132.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-03-18 23:07:20.120148286 +0000 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-03-18 23:07:20.120148286 +0000 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-03-18 23:09:06.503317791 +0000 2024-03-18T23:09:17.553 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-03-18 23:07:20.120148286 +0000 2024-03-18T23:09:17.553 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-18T23:09:17.553 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-18T23:09:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:17 smithi144 ceph-mon[66781]: [18/Mar/2024:23:09:16] ENGINE Bus STARTING 2024-03-18T23:09:17.629 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-18T23:09:17.684 INFO:tasks.workunit.client.0.smithi132.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-18T23:09:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:17 smithi196 ceph-mon[67089]: [18/Mar/2024:23:09:16] ENGINE Bus STARTING 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: [18/Mar/2024:23:09:16] ENGINE Serving on http://172.21.15.132:8765 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: [18/Mar/2024:23:09:16] ENGINE Serving on https://172.21.15.132:7150 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: [18/Mar/2024:23:09:16] ENGINE Bus STARTED 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:18 smithi144 ceph-mon[66781]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T23:09:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: [18/Mar/2024:23:09:16] ENGINE Serving on http://172.21.15.132:8765 2024-03-18T23:09:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: [18/Mar/2024:23:09:16] ENGINE Serving on https://172.21.15.132:7150 2024-03-18T23:09:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: [18/Mar/2024:23:09:16] ENGINE Bus STARTED 2024-03-18T23:09:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:18 smithi132 ceph-mon[32020]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T23:09:18.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: [18/Mar/2024:23:09:16] ENGINE Serving on http://172.21.15.132:8765 2024-03-18T23:09:18.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: [18/Mar/2024:23:09:16] ENGINE Serving on https://172.21.15.132:7150 2024-03-18T23:09:18.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: [18/Mar/2024:23:09:16] ENGINE Bus STARTED 2024-03-18T23:09:18.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.625 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:18.626 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:18 smithi196 ceph-mon[67089]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:19.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:19 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:19.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:19.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:19 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:09:19.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:19.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:19 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:09:20.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.475 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:20.476 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:09:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:09:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:09:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:09:20.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:20.611 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:09:20.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:09:20.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi132:/etc/ceph/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:20.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:09:20.617 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:09:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:21 smithi144 ceph-mon[66781]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:21 smithi144 ceph-mon[66781]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:21 smithi144 ceph-mon[66781]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:21 smithi132 ceph-mon[32020]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:21 smithi132 ceph-mon[32020]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:21 smithi132 ceph-mon[32020]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:21 smithi196 ceph-mon[67089]: Updating smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:21 smithi196 ceph-mon[67089]: Updating smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:21 smithi196 ceph-mon[67089]: Updating smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T23:09:22.552 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:22 smithi132 ceph-mon[32020]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:22 smithi144 ceph-mon[66781]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:22.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:22 smithi196 ceph-mon[67089]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T23:09:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:24 smithi144 ceph-mon[66781]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-18T23:09:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:24 smithi132 ceph-mon[32020]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-18T23:09:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:24 smithi196 ceph-mon[67089]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-03-18T23:09:26.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:26 smithi144 ceph-mon[66781]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-18T23:09:26.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:26 smithi132 ceph-mon[32020]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-18T23:09:26.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:26 smithi196 ceph-mon[67089]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-18T23:09:28.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:28 smithi144 ceph-mon[66781]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-18T23:09:28.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:28 smithi132 ceph-mon[32020]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-18T23:09:28.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:28 smithi196 ceph-mon[67089]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-18T23:09:30.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:30 smithi196 ceph-mon[67089]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:30.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:30 smithi144 ceph-mon[66781]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:30.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:30 smithi132 ceph-mon[32020]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.649+0000 7fbb25e7e640 1 -- 172.21.15.132:0/1235915555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 msgr2=0x7fbb2010e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.649+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/1235915555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb2010e700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ca0 tx=0x7fbb1802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.649+0000 7fbb25e7e640 1 -- 172.21.15.132:0/1235915555 shutdown_connections 2024-03-18T23:09:30.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.649+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/1235915555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb2010e700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.649+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/1235915555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb20100ca0 0x7fbb20106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/1235915555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb201002f0 0x7fbb201006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- 172.21.15.132:0/1235915555 >> 172.21.15.132:0/1235915555 conn(0x7fbb20075a60 msgr2=0x7fbb20075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- 172.21.15.132:0/1235915555 shutdown_connections 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- 172.21.15.132:0/1235915555 wait complete. 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 Processor -- start 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- start start 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb20198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2019c8e0 0x7fbb2019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111c00 con 0x7fbb201002f0 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111a80 con 0x7fbb20106b50 2024-03-18T23:09:30.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.650+0000 7fbb25e7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20111d80 con 0x7fbb2019c8e0 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33770/0 (socket says 172.21.15.132:33770) 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2539844430 learned_addr learned my addr 172.21.15.132:0/2539844430 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2019c8e0 0x7fbb2019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb20198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2539844430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2019c8e0 msgr2=0x7fbb2019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2019c8e0 0x7fbb2019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2539844430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 msgr2=0x7fbb20198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb20198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2539844430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2019d3d0 con 0x7fbb201002f0 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb20198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000e9f0 tx=0x7fbb1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1cff9640 1 -- 172.21.15.132:0/2539844430 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000cd80 con 0x7fbb201002f0 2024-03-18T23:09:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2019d6c0 con 0x7fbb201002f0 2024-03-18T23:09:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a4eb0 con 0x7fbb201002f0 2024-03-18T23:09:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1cff9640 1 -- 172.21.15.132:0/2539844430 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb10005500 con 0x7fbb201002f0 2024-03-18T23:09:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.651+0000 7fbb1cff9640 1 -- 172.21.15.132:0/2539844430 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10010690 con 0x7fbb201002f0 2024-03-18T23:09:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.653+0000 7fbb1cff9640 1 -- 172.21.15.132:0/2539844430 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 137467+0+0 (secure 0 0 0) 0x7fbb100046c0 con 0x7fbb201002f0 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.654+0000 7fbb1cff9640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 0x7fbb080a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.654+0000 7fbb28a64740 1 Processor -- start 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.654+0000 7fbb28a64740 1 -- start start 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.654+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 0x7fbb080a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1cff9640 1 -- 172.21.15.132:0/2539844430 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbb100cf250 con 0x7fbb201002f0 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 0x564db49e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x564db4943be0 con 0x564db49e0810 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db48ec960 con 0x564db49e95e0 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb28a64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564db493fc80 con 0x564db49dfef0 2024-03-18T23:09:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:54724/0 (socket says 172.21.15.132:54724) 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 -- 172.21.15.132:0/513842026 learned_addr learned my addr 172.21.15.132:0/513842026 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1effd640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 0x564db49e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 0x7fbb080a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb200fd6a0 tx=0x7fbb14009f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 -- 172.21.15.132:0/513842026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 msgr2=0x564db49e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 -- 172.21.15.132:0/513842026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 msgr2=0x564db49e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 0x564db49e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 -- 172.21.15.132:0/513842026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db48e3d20 con 0x564db49e95e0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1effd640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 0x564db49e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.655+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb1802c700 tx=0x7fbb18005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7701e90c622bbe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb05ffb640 1 -- 172.21.15.132:0/513842026 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18043070 con 0x564db49e95e0 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb05ffb640 1 -- 172.21.15.132:0/513842026 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1802cca0 con 0x564db49e95e0 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb05ffb640 1 -- 172.21.15.132:0/513842026 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18039040 con 0x564db49e95e0 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 -- 172.21.15.132:0/513842026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 msgr2=0x564db49eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb1802c700 tx=0x7fbb18005f80 comp rx=0 tx=0).stop 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 -- 172.21.15.132:0/513842026 shutdown_connections 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e95e0 0x564db49eb9d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e0810 0x564db49e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 --2- 172.21.15.132:0/513842026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 -- 172.21.15.132:0/513842026 >> 172.21.15.132:0/513842026 conn(0x564db49da810 msgr2=0x564db49dabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 -- 172.21.15.132:0/513842026 shutdown_connections 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.656+0000 7fbb28a64740 1 -- 172.21.15.132:0/513842026 wait complete. 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 -- start start 2024-03-18T23:09:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 Processor -- start 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x564db49ec1b0 con 0x564db49e95e0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db48ffdb0 con 0x564db49e0810 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564db49ec330 con 0x564db49dfef0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb28a64740 10 client.0 _ll_register_callbacks cb 0x564db4904e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:36118/0 (socket says 172.21.15.132:36118) 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2366243506 learned_addr learned my addr 172.21.15.132:0/2366243506 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.657+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 msgr2=0x564db49e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2366243506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 msgr2=0x564db49e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/2366243506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db48fec60 con 0x564db49dfef0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000e9c0 tx=0x7fbb10005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a628a114522e8368 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100b3070 con 0x564db49dfef0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1802cca0 con 0x564db49dfef0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbb10097de0 con 0x564db49dfef0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb100a3f10 con 0x564db49dfef0 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb28a64740 10 client.25228 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564db4658210 con 0x564db49dfef0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.658+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1802cca0 con 0x564db49dfef0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb100afb50 con 0x564db49dfef0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 10 client.25228 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 20 client.25228 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db49f9450 0x564db49fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 4 client.25228 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db4951120 con 0x564db49f9450 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db49f9450 0x564db49fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbafeffd640 10 client.25228 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.659+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db49f9450 0x564db49fb880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c1cbe01d592c78 server_cookie=185de94719855fe in_seq=0 out_seq=0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbafeffd640 1 -- 172.21.15.132:0/2366243506 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564db4951120 con 0x564db49f9450 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbafeffd640 10 client.25228 handle_command_reply: tid=0 2024-03-18T23:09:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 client.25228 shutdown 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 msgr2=0x564db49e8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000e9c0 tx=0x7fbb10005770 comp rx=0 tx=0).stop 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 shutdown_connections 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db49f9450 0x564db49fb880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2366243506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49e8f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbafeffd640 0 client.25228 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.660+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 >> 172.21.15.132:0/2366243506 conn(0x564db49da810 msgr2=0x564db49d9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.661+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 shutdown_connections 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.661+0000 7fbb28a64740 1 -- 172.21.15.132:0/2366243506 wait complete. 2024-03-18T23:09:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.661+0000 7fbb28a64740 20 client.25228 trim_cache size 0 max 16384 2024-03-18T23:09:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 Processor -- start 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 -- start start 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49e0810 0x564db4a01190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c010f40 con 0x564db49e95e0 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00e810 con 0x564db49dfef0 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb28a64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00c660 con 0x564db49e0810 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33790/0 (socket says 172.21.15.132:33790) 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49e0810 0x564db4a01190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:54740/0 (socket says 172.21.15.132:54740) 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.676+0000 7fbb1ffff640 1 -- 172.21.15.132:0/3170156334 learned_addr learned my addr 172.21.15.132:0/3170156334 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/3170156334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49e0810 msgr2=0x564db4a01190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49e0810 0x564db4a01190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/3170156334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 msgr2=0x564db49f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 -- 172.21.15.132:0/3170156334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db4724620 con 0x564db49dfef0 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb10097bd0 tx=0x7fbb1009b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936b839ca16f837f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb057fa640 1 -- 172.21.15.132:0/3170156334 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100b3040 con 0x564db49dfef0 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb057fa640 1 -- 172.21.15.132:0/3170156334 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb100ae070 con 0x564db49dfef0 2024-03-18T23:09:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.677+0000 7fbb057fa640 1 -- 172.21.15.132:0/3170156334 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10097de0 con 0x564db49dfef0 2024-03-18T23:09:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- 172.21.15.132:0/3170156334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 msgr2=0x564db49e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb10097bd0 tx=0x7fbb1009b670 comp rx=0 tx=0).stop 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- 172.21.15.132:0/3170156334 shutdown_connections 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49e0810 0x564db4a01190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- 172.21.15.132:0/3170156334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49dfef0 0x564db49e1750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- 172.21.15.132:0/3170156334 >> 172.21.15.132:0/3170156334 conn(0x564db49da810 msgr2=0x564db49e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- 172.21.15.132:0/3170156334 shutdown_connections 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- 172.21.15.132:0/3170156334 wait complete. 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- start start 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 Processor -- start 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00ccd0 con 0x564db49e95e0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00e140 con 0x564db49e0810 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.678+0000 7fbb28a64740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00d3a0 con 0x564db49dfef0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb28a64740 10 client.0 _ll_register_callbacks cb 0x7fbb100973d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33804/0 (socket says 172.21.15.132:33804) 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 -- 172.21.15.132:0/2299976934 learned_addr learned my addr 172.21.15.132:0/2299976934 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 -- 172.21.15.132:0/2299976934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 msgr2=0x564db49f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 -- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 msgr2=0x564db49ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 -- 172.21.15.132:0/2299976934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db49ed600 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1effd640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb1ffff640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ca0 tx=0x7fbb18003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f862513c1e99f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18039040 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fbb18048650 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb100ae070 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbb18043040 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb28a64740 10 client.15390 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.679+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564db4a5f520 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb100ae070 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbb1803fab0 con 0x564db49e95e0 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 10 client.15390 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 20 client.15390 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db4a5f800 0x564db4a61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 4 client.15390 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564db494b470 con 0x564db4a5f800 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db4a5f800 0x564db4a61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbafeffd640 10 client.15390 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.680+0000 7fbb1f7fe640 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db4a5f800 0x564db4a61bf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae813f06949f4b91 server_cookie=8953883df0df7454 in_seq=0 out_seq=0 2024-03-18T23:09:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbafeffd640 1 -- 172.21.15.132:0/2299976934 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564db494b470 con 0x564db4a5f800 2024-03-18T23:09:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbafeffd640 10 client.15390 handle_command_reply: tid=0 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 client.15390 shutdown 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 msgr2=0x564db49ed0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ca0 tx=0x7fbb18003970 comp rx=0 tx=0).stop 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 shutdown_connections 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x564db4a5f800 0x564db4a61bf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db49e95e0 0x564db49ed0c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db49e0810 0x564db49ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 --2- 172.21.15.132:0/2299976934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564db49dfef0 0x564db49f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 >> 172.21.15.132:0/2299976934 conn(0x564db49da810 msgr2=0x564db49e18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.690+0000 7fbafeffd640 0 client.15390 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 shutdown_connections 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb28a64740 1 -- 172.21.15.132:0/2299976934 wait complete. 2024-03-18T23:09:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb28a64740 20 client.15390 trim_cache size 0 max 16384 2024-03-18T23:09:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:09:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:09:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:09:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 msgr2=0x7fbb080a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 0x7fbb080a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb200fd6a0 tx=0x7fbb14009f90 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 msgr2=0x7fbb20198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000e9f0 tx=0x7fbb1000edf0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 shutdown_connections 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbb080a2ab0 0x7fbb080a4f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb2019c8e0 0x7fbb2019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20106b50 0x7fbb20198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 --2- 172.21.15.132:0/2539844430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb201002f0 0x7fbb20198020 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 >> 172.21.15.132:0/2539844430 conn(0x7fbb20075a60 msgr2=0x7fbb200fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 shutdown_connections 2024-03-18T23:09:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:30.691+0000 7fbb25e7e640 1 -- 172.21.15.132:0/2539844430 wait complete. 2024-03-18T23:09:32.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:32 smithi144 ceph-mon[66781]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:32.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:32 smithi132 ceph-mon[32020]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:32.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:32 smithi196 ceph-mon[67089]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-18T23:09:33.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:33 smithi144 ceph-mon[66781]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-03-18T23:09:33.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:33 smithi132 ceph-mon[32020]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-03-18T23:09:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:33 smithi196 ceph-mon[67089]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-03-18T23:09:36.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:36 smithi144 ceph-mon[66781]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:36.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:36 smithi132 ceph-mon[32020]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:36.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:36 smithi196 ceph-mon[67089]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:38.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:38 smithi144 ceph-mon[66781]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:38.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:38 smithi132 ceph-mon[32020]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:38.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:38 smithi196 ceph-mon[67089]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:40.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:40 smithi144 ceph-mon[66781]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:40.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:40 smithi132 ceph-mon[32020]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:40 smithi196 ceph-mon[67089]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:41 smithi144 ceph-mon[66781]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:41 smithi132 ceph-mon[32020]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:41 smithi196 ceph-mon[67089]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-18T23:09:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 -- 172.21.15.132:0/1007238320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e0106ba0 msgr2=0x7f16e010e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 --2- 172.21.15.132:0/1007238320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e0106ba0 0x7f16e010e700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009c10 tx=0x7f16d802bef0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 -- 172.21.15.132:0/1007238320 shutdown_connections 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 --2- 172.21.15.132:0/1007238320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e0106ba0 0x7f16e010e700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 --2- 172.21.15.132:0/1007238320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e0100940 0x7f16e0106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 --2- 172.21.15.132:0/1007238320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e00fff90 0x7f16e0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 -- 172.21.15.132:0/1007238320 >> 172.21.15.132:0/1007238320 conn(0x7f16e0076cb0 msgr2=0x7f16e00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 -- 172.21.15.132:0/1007238320 shutdown_connections 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.824+0000 7f16e74bf640 1 -- 172.21.15.132:0/1007238320 wait complete. 2024-03-18T23:09:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 Processor -- start 2024-03-18T23:09:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 -- start start 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 0x7f16e019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111af0 con 0x7f16e0106ba0 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111970 con 0x7f16e00fff90 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e525b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e74bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0111c70 con 0x7f16e019c7f0 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e525b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:51572/0 (socket says 172.21.15.132:51572) 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e525b640 1 -- 172.21.15.132:0/46271244 learned_addr learned my addr 172.21.15.132:0/46271244 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 0x7f16e019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.825+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36170/0 (socket says 172.21.15.132:36170) 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 -- 172.21.15.132:0/46271244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 msgr2=0x7f16e0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 -- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 msgr2=0x7f16e0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 -- 172.21.15.132:0/46271244 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e019d2e0 con 0x7f16e019c7f0 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e525b640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 0x7f16e019cca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16d802c430 tx=0x7f16d80336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16d67fc640 1 -- 172.21.15.132:0/46271244 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8043020 con 0x7f16e019c7f0 2024-03-18T23:09:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16d67fc640 1 -- 172.21.15.132:0/46271244 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d802ca80 con 0x7f16e019c7f0 2024-03-18T23:09:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e019d570 con 0x7f16e019c7f0 2024-03-18T23:09:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.826+0000 7f16d67fc640 1 -- 172.21.15.132:0/46271244 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d803e6b0 con 0x7f16e019c7f0 2024-03-18T23:09:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.827+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a4d90 con 0x7f16e019c7f0 2024-03-18T23:09:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.829+0000 7f16d67fc640 1 -- 172.21.15.132:0/46271244 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) ==== 137467+0+0 (secure 0 0 0) 0x7f16d8046020 con 0x7f16e019c7f0 2024-03-18T23:09:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 Processor -- start 2024-03-18T23:09:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 -- start start 2024-03-18T23:09:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e71a10 0x556325e71df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e81de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556325dd5be0 con 0x556325e72330 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556325d7e960 con 0x556325e7f9f0 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16ea181740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556325dd1c80 con 0x556325e71a10 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16d67fc640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 0x7f16cc0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.830+0000 7f16e525b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e71a10 0x556325e71df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36182/0 (socket says 172.21.15.132:36182) 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 -- 172.21.15.132:0/1900710452 learned_addr learned my addr 172.21.15.132:0/1900710452 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e81de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16d67fc640 1 -- 172.21.15.132:0/46271244 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f16d80f0e70 con 0x7f16e019c7f0 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 0x7f16cc0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 -- 172.21.15.132:0/1900710452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e71a10 msgr2=0x556325e71df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e71a10 0x556325e71df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 -- 172.21.15.132:0/1900710452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 msgr2=0x556325e81de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e81de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 -- 172.21.15.132:0/1900710452 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556325d75d20 con 0x556325e72330 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e525b640 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009dc0 tx=0x7f16dc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e903328e7dcf49d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16e5a5c640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 0x7f16cc0a4f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ee6a0 tx=0x7f16d802bef0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16cb7fe640 1 -- 172.21.15.132:0/1900710452 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc012450 con 0x556325e72330 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16cb7fe640 1 -- 172.21.15.132:0/1900710452 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc012ad0 con 0x556325e72330 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.831+0000 7f16cb7fe640 1 -- 172.21.15.132:0/1900710452 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00fca0 con 0x556325e72330 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 -- 172.21.15.132:0/1900710452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 msgr2=0x556325e7f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009dc0 tx=0x7f16dc0117b0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 -- 172.21.15.132:0/1900710452 shutdown_connections 2024-03-18T23:09:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e81de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e72330 0x556325e7f4b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 --2- 172.21.15.132:0/1900710452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e71a10 0x556325e71df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.832+0000 7f16ea181740 1 -- 172.21.15.132:0/1900710452 >> 172.21.15.132:0/1900710452 conn(0x556325e6c810 msgr2=0x556325e6cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.833+0000 7f16ea181740 1 -- 172.21.15.132:0/1900710452 shutdown_connections 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.833+0000 7f16ea181740 1 -- 172.21.15.132:0/1900710452 wait complete. 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.833+0000 7f16ea181740 1 -- start start 2024-03-18T23:09:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.833+0000 7f16ea181740 1 Processor -- start 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 0x556325e8ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556325e825b0 con 0x556325e71a10 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556325d91db0 con 0x556325e7d240 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556325e82730 con 0x556325e7f9f0 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60250/0 (socket says 172.21.15.132:60250) 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e7c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36198/0 (socket says 172.21.15.132:36198) 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 -- 172.21.15.132:0/2802043320 learned_addr learned my addr 172.21.15.132:0/2802043320 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16ea181740 10 client.0 _ll_register_callbacks cb 0x556325d96e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 0x556325e8ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 -- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 msgr2=0x556325e8ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 0x556325e8ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 -- 172.21.15.132:0/2802043320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 msgr2=0x556325e7c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e7c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 -- 172.21.15.132:0/2802043320 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556325d90c60 con 0x556325e7f9f0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 0x556325e8ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16e525b640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f16dc01c040 tx=0x7f16dc009b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2867ab8cbae96bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.834+0000 7f16c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc028070 con 0x556325e7f9f0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f16dc00fdf0 con 0x556325e7f9f0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc012ad0 con 0x556325e7f9f0 2024-03-18T23:09:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16dc034050 con 0x556325e7f9f0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16ea181740 10 client.25246 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556325aea210 con 0x556325e7f9f0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.835+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc005040 con 0x556325e7f9f0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f16dc0046c0 con 0x556325e7f9f0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 10 client.25246 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 10 client.25246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 20 client.25246 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325e920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 4 client.25246 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f16dc012ad0 con 0x556325e8fd00 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325e920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325e920f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97807b9c8a6737af server_cookie=25f0b69425032cff in_seq=0 out_seq=0 2024-03-18T23:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.836+0000 7f16c8ff9640 10 client.25246 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2802043320 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f16dc012ad0 con 0x556325e8fd00 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16c8ff9640 10 client.25246 handle_command_reply: tid=0 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 client.25246 shutdown 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 msgr2=0x556325e7cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f16dc01c040 tx=0x7f16dc009b10 comp rx=0 tx=0).stop 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 shutdown_connections 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325e920f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7d240 0x556325e8ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7f9f0 0x556325e7cd00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16c8ff9640 0 client.25246 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 --2- 172.21.15.132:0/2802043320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 >> 172.21.15.132:0/2802043320 conn(0x556325e6c810 msgr2=0x556325e6e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 shutdown_connections 2024-03-18T23:09:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 1 -- 172.21.15.132:0/2802043320 wait complete. 2024-03-18T23:09:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.837+0000 7f16ea181740 20 client.25246 trim_cache size 0 max 16384 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.865+0000 7f16ea181740 1 Processor -- start 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.865+0000 7f16ea181740 1 -- start start 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e8b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01ada70 con 0x556325e71a10 2024-03-18T23:09:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01ab340 con 0x556325e7f9f0 2024-03-18T23:09:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16ea181740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01a91b0 con 0x556325e7d240 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60264/0 (socket says 172.21.15.132:60264) 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 -- 172.21.15.132:0/2269863101 learned_addr learned my addr 172.21.15.132:0/2269863101 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 -- 172.21.15.132:0/2269863101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 msgr2=0x556325e8b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e8b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e8b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 -- 172.21.15.132:0/2269863101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 msgr2=0x556325e97750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e97750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e525b640 1 -- 172.21.15.132:0/2269863101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556325bb6620 con 0x556325e7d240 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.866+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:09:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.867+0000 7f16e525b640 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0092e0 tx=0x7f16dc009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9587332569d808 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.867+0000 7f16caffd640 1 -- 172.21.15.132:0/2269863101 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03d070 con 0x556325e7d240 2024-03-18T23:09:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.867+0000 7f16caffd640 1 -- 172.21.15.132:0/2269863101 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc012ad0 con 0x556325e7d240 2024-03-18T23:09:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.867+0000 7f16caffd640 1 -- 172.21.15.132:0/2269863101 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc028070 con 0x556325e7d240 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- 172.21.15.132:0/2269863101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 msgr2=0x556325e8b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0092e0 tx=0x7f16dc009c10 comp rx=0 tx=0).stop 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- 172.21.15.132:0/2269863101 shutdown_connections 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325e8b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e8b390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 --2- 172.21.15.132:0/2269863101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- 172.21.15.132:0/2269863101 >> 172.21.15.132:0/2269863101 conn(0x556325e6c810 msgr2=0x556325e6bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- 172.21.15.132:0/2269863101 shutdown_connections 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- 172.21.15.132:0/2269863101 wait complete. 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 -- start start 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.868+0000 7f16ea181740 1 Processor -- start 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e7a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325ef4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01a9800 con 0x556325e71a10 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01aac70 con 0x556325e7f9f0 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01a9ed0 con 0x556325e7d240 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16ea181740 10 client.0 _ll_register_callbacks cb 0x7f16d0013640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e525b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e7a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e4a5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36214/0 (socket says 172.21.15.132:36214) 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e4a5a640 1 -- 172.21.15.132:0/2606334389 learned_addr learned my addr 172.21.15.132:0/2606334389 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e525b640 1 -- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 msgr2=0x556325ef4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e525b640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325ef4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.869+0000 7f16e525b640 1 -- 172.21.15.132:0/2606334389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 msgr2=0x556325e79b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16e525b640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16e525b640 1 -- 172.21.15.132:0/2606334389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556325ef52c0 con 0x556325e7d240 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16e525b640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e7a040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556325e7f730 tx=0x7f16dc00ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc0866ffe76969 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03d070 con 0x556325e7d240 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f16dc012660 con 0x556325e7d240 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16dc012ad0 con 0x556325e7d240 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16dc028070 con 0x556325e7d240 2024-03-18T23:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16ea181740 10 client.25252 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.870+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556325ef65d0 con 0x556325e7d240 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc012ad0 con 0x556325e7d240 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f16dc004030 con 0x556325e7d240 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 10 client.25252 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 10 client.25252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 20 client.25252 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325ef8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 4 client.25252 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556325de3120 con 0x556325e8fd00 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325ef8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.871+0000 7f16c8ff9640 10 client.25252 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.872+0000 7f16e4a5a640 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325ef8bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c930d03e22b1db5 server_cookie=f17cd82cb0b90c34 in_seq=0 out_seq=0 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16c8ff9640 1 -- 172.21.15.132:0/2606334389 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556325de3120 con 0x556325e8fd00 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16c8ff9640 10 client.25252 handle_command_reply: tid=0 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 client.25252 shutdown 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 msgr2=0x556325e7a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e7a040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556325e7f730 tx=0x7f16dc00ffa0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 shutdown_connections 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556325e8fd00 0x556325ef8bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556325e7f9f0 0x556325ef4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16c8ff9640 0 client.25252 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556325e7d240 0x556325e7a040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 --2- 172.21.15.132:0/2606334389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556325e71a10 0x556325e79b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 >> 172.21.15.132:0/2606334389 conn(0x556325e6c810 msgr2=0x556325e77450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 shutdown_connections 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 1 -- 172.21.15.132:0/2606334389 wait complete. 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.875+0000 7f16ea181740 20 client.25252 trim_cache size 0 max 16384 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:09:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:09:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 msgr2=0x7f16cc0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 0x7f16cc0a4f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ee6a0 tx=0x7f16d802bef0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 msgr2=0x7f16e019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 0x7f16e019cca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16d802c430 tx=0x7f16d80336b0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 shutdown_connections 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f16cc0a2ab0 0x7f16cc0a4f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16e019c7f0 0x7f16e019cca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16e0106ba0 0x7f16e0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 --2- 172.21.15.132:0/46271244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e00fff90 0x7f16e0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 >> 172.21.15.132:0/46271244 conn(0x7f16e0076cb0 msgr2=0x7f16e0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 shutdown_connections 2024-03-18T23:09:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:43.876+0000 7f16e74bf640 1 -- 172.21.15.132:0/46271244 wait complete. 2024-03-18T23:09:44.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:44 smithi144 ceph-mon[66781]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:44.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:44 smithi132 ceph-mon[32020]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:44.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:44 smithi196 ceph-mon[67089]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T23:09:46.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:46 smithi144 ceph-mon[66781]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:46.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:46 smithi132 ceph-mon[32020]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:46.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:46 smithi196 ceph-mon[67089]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:09:47.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:47 smithi144 ceph-mon[66781]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-18T23:09:47.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:47 smithi132 ceph-mon[32020]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-18T23:09:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:47 smithi196 ceph-mon[67089]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-18T23:09:50.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:50 smithi144 ceph-mon[66781]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:50.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:50 smithi132 ceph-mon[32020]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:50.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:50 smithi196 ceph-mon[67089]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:52.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:52 smithi144 ceph-mon[66781]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:52.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:52 smithi132 ceph-mon[32020]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:52.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:52 smithi196 ceph-mon[67089]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-18T23:09:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:53 smithi144 ceph-mon[66781]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:53 smithi132 ceph-mon[32020]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:53 smithi196 ceph-mon[67089]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:55.447 INFO:tasks.workunit.client.0.smithi132.stderr:Note: switching to 'e48b1f6c0608b2615e1890cc46286c3877d6e910'. 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: git switch -c 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr:Or undo this operation with: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.448 INFO:tasks.workunit.client.0.smithi132.stderr: git switch - 2024-03-18T23:09:55.449 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.449 INFO:tasks.workunit.client.0.smithi132.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T23:09:55.449 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-03-18T23:09:55.449 INFO:tasks.workunit.client.0.smithi132.stderr:HEAD is now at e48b1f6c060 testfix 2024-03-18T23:09:55.452 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-18T23:09:55.519 INFO:tasks.workunit.client.0.smithi132.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T23:09:55.521 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T23:09:55.521 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T23:09:56.249 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T23:09:56.319 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T23:09:56.360 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T23:09:56.362 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T23:09:56.362 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T23:09:56.401 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T23:09:56.404 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:09:56.404 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-18T23:09:56.461 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-03-18T23:09:56.462 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-03-18T23:09:56.462 DEBUG:teuthology.orchestra.run.smithi132:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-18T23:09:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-18T23:09:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-03-18T23:09:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-03-18T23:09:56.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:56 smithi144 ceph-mon[66781]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-18T23:09:56.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:56 smithi132 ceph-mon[32020]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-18T23:09:56.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:56 smithi196 ceph-mon[67089]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-18T23:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- 172.21.15.132:0/745435631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730102190 msgr2=0x7f573010e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 --2- 172.21.15.132:0/745435631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730102190 0x7f573010e6a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5728009ca0 tx=0x7f572802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- 172.21.15.132:0/745435631 shutdown_connections 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 --2- 172.21.15.132:0/745435631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730102190 0x7f573010e6a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 --2- 172.21.15.132:0/745435631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57301017d0 0x7f5730101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 --2- 172.21.15.132:0/745435631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301077e0 0x7f5730107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- 172.21.15.132:0/745435631 >> 172.21.15.132:0/745435631 conn(0x7f57300fd2e0 msgr2=0x7f57300ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- 172.21.15.132:0/745435631 shutdown_connections 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- 172.21.15.132:0/745435631 wait complete. 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 Processor -- start 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.968+0000 7f57386bf640 1 -- start start 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5730102190 0x7f5730198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 0x7f573019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5730111bf0 con 0x7f57301017d0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5730111a70 con 0x7f57301077e0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5730111d70 con 0x7f5730102190 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f5736c5c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 0x7f573019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58056/0 (socket says 172.21.15.132:58056) 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 -- 172.21.15.132:0/2822961129 learned_addr learned my addr 172.21.15.132:0/2822961129 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5730102190 0x7f5730198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 -- 172.21.15.132:0/2822961129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5730102190 msgr2=0x7f5730198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5730102190 0x7f5730198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 -- 172.21.15.132:0/2822961129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 msgr2=0x7f573019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 0x7f573019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 -- 172.21.15.132:0/2822961129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573019cfc0 con 0x7f57301017d0 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f5736c5c640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 0x7f573019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f573645b640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f572000e9f0 tx=0x7f572000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57277fe640 1 -- 172.21.15.132:0/2822961129 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572000cd80 con 0x7f57301017d0 2024-03-18T23:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573019d2b0 con 0x7f57301017d0 2024-03-18T23:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57277fe640 1 -- 172.21.15.132:0/2822961129 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5720010910 con 0x7f57301017d0 2024-03-18T23:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57277fe640 1 -- 172.21.15.132:0/2822961129 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200188f0 con 0x7f57301017d0 2024-03-18T23:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.969+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57301a4b40 con 0x7f57301017d0 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.971+0000 7f57277fe640 1 -- 172.21.15.132:0/2822961129 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 23) ==== 137467+0+0 (secure 0 0 0) 0x7f5720010ac0 con 0x7f57301017d0 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 Processor -- start 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 -- start start 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 0x5650617810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x5650617839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5650616dbbe0 con 0x565061777ef0 2024-03-18T23:09:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565061684960 con 0x5650617815e0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573b382740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5650616d7c80 con 0x565061778810 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f5736c5c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f57277fe640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 0x7f571c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f5736c5c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58058/0 (socket says 172.21.15.132:58058) 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f5736c5c640 1 -- 172.21.15.132:0/1332087998 learned_addr learned my addr 172.21.15.132:0/1332087998 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x5650617839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f573645b640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 0x5650617810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.972+0000 7f5736c5c640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 0x7f571c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 -- 172.21.15.132:0/1332087998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 msgr2=0x5650617810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f57277fe640 1 -- 172.21.15.132:0/2822961129 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f57200cf310 con 0x7f57301017d0 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 0x5650617810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 -- 172.21.15.132:0/1332087998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 msgr2=0x5650617782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 -- 172.21.15.132:0/1332087998 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506167bd20 con 0x5650617815e0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5736c5c640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f573645b640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 0x5650617810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x5650617839d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f572c009bd0 tx=0x7f572c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5041668eaaa02d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5736c5c640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 0x7f571c0a4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5728006010 tx=0x7f572802c0f0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5724ff9640 1 -- 172.21.15.132:0/1332087998 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c004230 con 0x5650617815e0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5724ff9640 1 -- 172.21.15.132:0/1332087998 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c009e40 con 0x5650617815e0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.973+0000 7f5724ff9640 1 -- 172.21.15.132:0/1332087998 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c004d50 con 0x5650617815e0 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- 172.21.15.132:0/1332087998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 msgr2=0x5650617839d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x5650617839d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f572c009bd0 tx=0x7f572c011810 comp rx=0 tx=0).stop 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- 172.21.15.132:0/1332087998 shutdown_connections 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x5650617839d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061778810 0x5650617810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- 172.21.15.132:0/1332087998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061777ef0 0x5650617782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- 172.21.15.132:0/1332087998 >> 172.21.15.132:0/1332087998 conn(0x565061772810 msgr2=0x565061772bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- 172.21.15.132:0/1332087998 shutdown_connections 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- 172.21.15.132:0/1332087998 wait complete. 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- start start 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 Processor -- start 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x565061780f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x56506177e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x565061784210 con 0x565061778810 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565061697db0 con 0x565061777ef0 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x565061784390 con 0x5650617815e0 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573b382740 10 client.0 _ll_register_callbacks cb 0x56506169ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.974+0000 7f573645b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x565061780f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58066/0 (socket says 172.21.15.132:58066) 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 -- 172.21.15.132:0/2931043656 learned_addr learned my addr 172.21.15.132:0/2931043656 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5736c5c640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x56506177e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 -- 172.21.15.132:0/2931043656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 msgr2=0x56506177e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x56506177e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 -- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 msgr2=0x565061780f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x565061780f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 -- 172.21.15.132:0/2931043656 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565061696c60 con 0x565061778810 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f573645b640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x565061780f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f572c011d10 tx=0x7f572c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5505cfa48cf090dc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c027020 con 0x565061778810 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f572c0038b0 con 0x565061778810 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650616e9120 con 0x565061778810 2024-03-18T23:09:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f572c032050 con 0x565061778810 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f573b382740 10 client.15420 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650613f0210 con 0x565061778810 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.975+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5650616e9120 con 0x565061778810 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f572c018070 con 0x565061778810 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 10 client.15420 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 20 client.15420 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617913d0 0x5650617937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 4 client.15420 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650616e3470 con 0x5650617913d0 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573645b640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617913d0 0x5650617937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f5719ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.976+0000 7f573645b640 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617913d0 0x5650617937c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bb4cfbd4899f40 server_cookie=564b031c5447dada in_seq=0 out_seq=0 2024-03-18T23:09:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f5719ffb640 1 -- 172.21.15.132:0/2931043656 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650616e3470 con 0x5650617913d0 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f5719ffb640 10 client.15420 handle_command_reply: tid=0 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 client.15420 shutdown 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 msgr2=0x56506177e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f572c011d10 tx=0x7f572c0098b0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 shutdown_connections 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617913d0 0x5650617937c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x56506177e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x56506177e2d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 --2- 172.21.15.132:0/2931043656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x565061780f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f5719ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 >> 172.21.15.132:0/2931043656 conn(0x565061772810 msgr2=0x565061771ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 shutdown_connections 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 1 -- 172.21.15.132:0/2931043656 wait complete. 2024-03-18T23:09:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.977+0000 7f573b382740 20 client.15420 trim_cache size 0 max 16384 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.991+0000 7f573b382740 1 Processor -- start 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.991+0000 7f573b382740 1 -- start start 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.991+0000 7f573b382740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x56506178cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.991+0000 7f573b382740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f573b382740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x565061780a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f573b382740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f57301afc60 con 0x565061778810 2024-03-18T23:09:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f573b382740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57301ad530 con 0x565061777ef0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f573b382740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57301ab380 con 0x5650617815e0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58078/0 (socket says 172.21.15.132:58078) 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 -- 172.21.15.132:0/2082850792 learned_addr learned my addr 172.21.15.132:0/2082850792 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f573645b640 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x56506178cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5736c5c640 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x565061780a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 -- 172.21.15.132:0/2082850792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 msgr2=0x565061780a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x565061780a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 -- 172.21.15.132:0/2082850792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 msgr2=0x56506178cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x56506178cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 -- 172.21.15.132:0/2082850792 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650614bc620 con 0x565061778810 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f5735c5a640 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f572c0058c0 tx=0x7f572c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b96c3847fe0319 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f571bfff640 1 -- 172.21.15.132:0/2082850792 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c027070 con 0x565061778810 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f571bfff640 1 -- 172.21.15.132:0/2082850792 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0238c0 con 0x565061778810 2024-03-18T23:09:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.992+0000 7f571bfff640 1 -- 172.21.15.132:0/2082850792 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c01e040 con 0x565061778810 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- 172.21.15.132:0/2082850792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 msgr2=0x565061798f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f572c0058c0 tx=0x7f572c005850 comp rx=0 tx=0).stop 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- 172.21.15.132:0/2082850792 shutdown_connections 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5650617815e0 0x565061780a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x565061798f80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- 172.21.15.132:0/2082850792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565061777ef0 0x56506178cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- 172.21.15.132:0/2082850792 >> 172.21.15.132:0/2082850792 conn(0x565061772810 msgr2=0x565061771ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- 172.21.15.132:0/2082850792 shutdown_connections 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- 172.21.15.132:0/2082850792 wait complete. 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- start start 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 Processor -- start 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061777ef0 0x565061789490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x565061784af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f57301ab9f0 con 0x565061778810 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57301ace60 con 0x5650617815e0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57301ac0c0 con 0x565061777ef0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f573b382740 10 client.0 _ll_register_callbacks cb 0x7f5720005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.993+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58090/0 (socket says 172.21.15.132:58090) 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 -- 172.21.15.132:0/1087752371 learned_addr learned my addr 172.21.15.132:0/1087752371 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5736c5c640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x565061784af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f573645b640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061777ef0 0x565061789490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 -- 172.21.15.132:0/1087752371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061777ef0 msgr2=0x565061789490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061777ef0 0x565061789490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 -- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 msgr2=0x565061784af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x565061784af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 -- 172.21.15.132:0/1087752371 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650617852f0 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5735c5a640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f572c011850 tx=0x7f572c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef33debab0174af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c027040 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f572c0235f0 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c0238c0 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f572c045050 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f573b382740 10 client.15426 fetch_fsmap learned FSMap version 21 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650617f71d0 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c0238c0 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.994+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f572c018050 con 0x565061778810 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 10 client.15426 fetch_fsmap finished waiting for FSMap version 21 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 20 client.15426 populate_metadata read hostname 'smithi132' 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617f7910 0x5650617f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 4 client.15426 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565061661770 con 0x5650617f7910 2024-03-18T23:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573645b640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617f7910 0x5650617f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f5719ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:56.995+0000 7f573645b640 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617f7910 0x5650617f9d30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b965aacc9ddcf7e server_cookie=37afb8f604721587 in_seq=0 out_seq=0 2024-03-18T23:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.003+0000 7f5719ffb640 1 -- 172.21.15.132:0/1087752371 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565061661770 con 0x5650617f7910 2024-03-18T23:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.003+0000 7f5719ffb640 10 client.15426 handle_command_reply: tid=0 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.003+0000 7f573b382740 1 client.15426 shutdown 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 msgr2=0x5650617899d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f572c011850 tx=0x7f572c004350 comp rx=0 tx=0).stop 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 shutdown_connections 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5650617f7910 0x5650617f9d30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650617815e0 0x565061784af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565061778810 0x5650617899d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f5719ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 --2- 172.21.15.132:0/1087752371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565061777ef0 0x565061789490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 >> 172.21.15.132:0/1087752371 conn(0x565061772810 msgr2=0x565061773a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 shutdown_connections 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 1 -- 172.21.15.132:0/1087752371 wait complete. 2024-03-18T23:09:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f573b382740 20 client.15426 trim_cache size 0 max 16384 2024-03-18T23:09:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:09:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:09:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:09:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:09:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 msgr2=0x7f571c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 0x7f571c0a4f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5728006010 tx=0x7f572802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 msgr2=0x7f5730197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f572000e9f0 tx=0x7f572000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 shutdown_connections 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f571c0a2ab0 0x7f571c0a4f70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57301077e0 0x7f573019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5730102190 0x7f5730198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 --2- 172.21.15.132:0/2822961129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57301017d0 0x7f5730197fd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 >> 172.21.15.132:0/2822961129 conn(0x7f57300fd2e0 msgr2=0x7f57300fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 shutdown_connections 2024-03-18T23:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:09:57.004+0000 7f57386bf640 1 -- 172.21.15.132:0/2822961129 wait complete. 2024-03-18T23:09:58.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:58 smithi144 ceph-mon[66781]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:58.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:58 smithi132 ceph-mon[32020]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:58.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:58 smithi196 ceph-mon[67089]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-18T23:09:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:09:59 smithi144 ceph-mon[66781]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-18T23:09:59.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:09:59 smithi132 ceph-mon[32020]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-18T23:09:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:09:59 smithi196 ceph-mon[67089]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-18T23:10:00.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:00 smithi144 ceph-mon[66781]: overall HEALTH_OK 2024-03-18T23:10:00.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-mon[32020]: overall HEALTH_OK 2024-03-18T23:10:00.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:00 smithi196 ceph-mon[67089]: overall HEALTH_OK 2024-03-18T23:10:01.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:10:00.705+0000 7fb4183d9700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:10:01.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:10:00.705+0000 7fb4183d9700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:10:01.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:10:00.705+0000 7fb4183d9700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:10:01.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:10:00.705+0000 7fb4183d9700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:10:01.116 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:10:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:10:00.705+0000 7fb4183d9700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T23:10:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:01 smithi144 ceph-mon[66781]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-18T23:10:01.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:01 smithi132 ceph-mon[32020]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-18T23:10:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:01 smithi196 ceph-mon[67089]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-18T23:10:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:03 smithi144 ceph-mon[66781]: mgrmap e24: x(active, since 47s), standbys: z, y 2024-03-18T23:10:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:03 smithi144 ceph-mon[66781]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-18T23:10:03.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:03 smithi132 ceph-mon[32020]: mgrmap e24: x(active, since 47s), standbys: z, y 2024-03-18T23:10:03.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:03 smithi132 ceph-mon[32020]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-18T23:10:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:03 smithi196 ceph-mon[67089]: mgrmap e24: x(active, since 47s), standbys: z, y 2024-03-18T23:10:03.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:03 smithi196 ceph-mon[67089]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 26 op/s 2024-03-18T23:10:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:05 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:10:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:05 smithi144 ceph-mon[66781]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-03-18T23:10:05.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:05 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:10:05.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:05 smithi132 ceph-mon[32020]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-03-18T23:10:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:05 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:10:05.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:05 smithi196 ceph-mon[67089]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-03-18T23:10:08.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:08 smithi144 ceph-mon[66781]: pgmap v29: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-03-18T23:10:08.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:08 smithi132 ceph-mon[32020]: pgmap v29: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-03-18T23:10:08.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:08 smithi196 ceph-mon[67089]: pgmap v29: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 30 op/s 2024-03-18T23:10:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:09 smithi144 ceph-mon[66781]: pgmap v30: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-03-18T23:10:09.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:09 smithi132 ceph-mon[32020]: pgmap v30: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-03-18T23:10:09.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:09 smithi196 ceph-mon[67089]: pgmap v30: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.101+0000 7ff00bdbc640 1 -- 172.21.15.132:0/2173683973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004104230 msgr2=0x7ff004104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/2173683973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004104230 0x7ff004104690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc02c180 comp rx=0 tx=0).stop 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 -- 172.21.15.132:0/2173683973 shutdown_connections 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/2173683973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff004104230 0x7ff004104690 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/2173683973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0040693e0 0x7ff004069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/2173683973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0041067e0 0x7ff004068ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 -- 172.21.15.132:0/2173683973 >> 172.21.15.132:0/2173683973 conn(0x7ff004077570 msgr2=0x7ff004077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 -- 172.21.15.132:0/2173683973 shutdown_connections 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 -- 172.21.15.132:0/2173683973 wait complete. 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 Processor -- start 2024-03-18T23:10:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.102+0000 7ff00bdbc640 1 -- start start 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 0x7ff00419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 0x7ff0041a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff004118130 con 0x7ff0041067e0 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff004117fb0 con 0x7ff0040693e0 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041182b0 con 0x7ff004104230 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:56154/0 (socket says 172.21.15.132:56154) 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009b58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 0x7ff00419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 -- 172.21.15.132:0/505116015 learned_addr learned my addr 172.21.15.132:0/505116015 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00a359640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 0x7ff0041a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 -- 172.21.15.132:0/505116015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 msgr2=0x7ff00419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 0x7ff00419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 -- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 msgr2=0x7ff0041a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 0x7ff0041a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 -- 172.21.15.132:0/505116015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0041a34c0 con 0x7ff004104230 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009b58640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 0x7ff00419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00a359640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 0x7ff0041a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff009357640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feff400ea60 tx=0x7feff400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7feffaffd640 1 -- 172.21.15.132:0/505116015 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff400cde0 con 0x7ff004104230 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0041a37b0 con 0x7ff004104230 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.103+0000 7feffaffd640 1 -- 172.21.15.132:0/505116015 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff40035f0 con 0x7ff004104230 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.104+0000 7feffaffd640 1 -- 172.21.15.132:0/505116015 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4010670 con 0x7ff004104230 2024-03-18T23:10:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.104+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041ab040 con 0x7ff004104230 2024-03-18T23:10:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.106+0000 7feffaffd640 1 -- 172.21.15.132:0/505116015 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 24) ==== 137524+0+0 (secure 0 0 0) 0x7feff40037a0 con 0x7ff004104230 2024-03-18T23:10:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 Processor -- start 2024-03-18T23:10:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 -- start start 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55bdf4444be0 con 0x55bdf44ea5e0 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bdf43ed960 con 0x55bdf44e1810 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00e99c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bdf4440c80 con 0x55bdf44e0ef0 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7feffaffd640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 0x7feff00a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff009357640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00a359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff009b58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00a359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:56160/0 (socket says 172.21.15.132:56160) 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff009357640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44ec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41960/0 (socket says 172.21.15.132:41960) 2024-03-18T23:10:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff00a359640 1 -- 172.21.15.132:0/1830605729 learned_addr learned my addr 172.21.15.132:0/1830605729 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.107+0000 7ff009b58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:54414/0 (socket says 172.21.15.132:54414) 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff00a359640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 0x7feff00a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7feffaffd640 1 -- 172.21.15.132:0/505116015 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feff40cf380 con 0x7ff004104230 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 -- 172.21.15.132:0/1830605729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 msgr2=0x55bdf44e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 -- 172.21.15.132:0/1830605729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 msgr2=0x55bdf44ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 -- 172.21.15.132:0/1830605729 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf43e4d20 con 0x55bdf44e1810 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff00a359640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 0x7feff00a4ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feffc037040 tx=0x7feffc005c50 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7ff009b58640 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff000009fd0 tx=0x7ff0000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24142c9140a3bd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7fefeffff640 1 -- 172.21.15.132:0/1830605729 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000004020 con 0x55bdf44e1810 2024-03-18T23:10:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7fefeffff640 1 -- 172.21.15.132:0/1830605729 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000005500 con 0x55bdf44e1810 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.108+0000 7fefeffff640 1 -- 172.21.15.132:0/1830605729 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000004d40 con 0x55bdf44e1810 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- 172.21.15.132:0/1830605729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 msgr2=0x55bdf44ea0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff000009fd0 tx=0x7ff0000118d0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- 172.21.15.132:0/1830605729 shutdown_connections 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e1810 0x55bdf44ea0a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 --2- 172.21.15.132:0/1830605729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- 172.21.15.132:0/1830605729 >> 172.21.15.132:0/1830605729 conn(0x55bdf44db810 msgr2=0x55bdf44dbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- 172.21.15.132:0/1830605729 shutdown_connections 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- 172.21.15.132:0/1830605729 wait complete. 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.109+0000 7ff00e99c740 1 -- start start 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 Processor -- start 2024-03-18T23:10:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf44e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55bdf44ed210 con 0x55bdf44ea5e0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bdf4400db0 con 0x55bdf44e0ef0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bdf44ed390 con 0x55bdf44e1810 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00e99c740 10 client.0 _ll_register_callbacks cb 0x55bdf4405e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff009b58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41976/0 (socket says 172.21.15.132:41976) 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff009357640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf44e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 -- 172.21.15.132:0/364600849 learned_addr learned my addr 172.21.15.132:0/364600849 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff009b58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:54428/0 (socket says 172.21.15.132:54428) 2024-03-18T23:10:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 -- 172.21.15.132:0/364600849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 msgr2=0x55bdf44e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf44e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 -- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 msgr2=0x55bdf44e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff00a359640 1 -- 172.21.15.132:0/364600849 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf43ffc60 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.110+0000 7ff009b58640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00a359640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feffc02c6c0 tx=0x7feffc02c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5019b7b0506fc348 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc039040 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feffc032a80 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff000005500 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feffc05c050 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 10 client.15444 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdf4159210 con 0x55bdf44ea5e0 2024-03-18T23:10:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000005500 con 0x55bdf44ea5e0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7feffc042420 con 0x55bdf44ea5e0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 10 client.15444 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 20 client.15444 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf44fa4a0 0x55bdf44fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 4 client.15444 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.111+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf4452120 con 0x55bdf44fa4a0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.112+0000 7ff009b58640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf44fa4a0 0x55bdf44fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.112+0000 7ff009b58640 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf44fa4a0 0x55bdf44fc8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9097e67b8703aff server_cookie=9f75b50fd3aff5bc in_seq=0 out_seq=0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.112+0000 7fefed7fa640 10 client.15444 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7fefed7fa640 1 -- 172.21.15.132:0/364600849 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bdf4452120 con 0x55bdf44fa4a0 2024-03-18T23:10:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7fefed7fa640 10 client.15444 handle_command_reply: tid=0 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 client.15444 shutdown 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 msgr2=0x55bdf44e7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feffc02c6c0 tx=0x7feffc02c180 comp rx=0 tx=0).stop 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 shutdown_connections 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf44fa4a0 0x55bdf44fc8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e7870 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf44e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 --2- 172.21.15.132:0/364600849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7fefed7fa640 0 client.15444 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 >> 172.21.15.132:0/364600849 conn(0x55bdf44db810 msgr2=0x55bdf44daab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 shutdown_connections 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 1 -- 172.21.15.132:0/364600849 wait complete. 2024-03-18T23:10:10.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.113+0000 7ff00e99c740 20 client.15444 trim_cache size 0 max 16384 2024-03-18T23:10:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 Processor -- start 2024-03-18T23:10:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 -- start start 2024-03-18T23:10:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf4502200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b62b0 con 0x55bdf44ea5e0 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b3b80 con 0x55bdf44e0ef0 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00e99c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b19d0 con 0x55bdf44e1810 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00a359640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00a359640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41978/0 (socket says 172.21.15.132:41978) 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff00a359640 1 -- 172.21.15.132:0/961204507 learned_addr learned my addr 172.21.15.132:0/961204507 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff009b58640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.128+0000 7ff009357640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf4502200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 -- 172.21.15.132:0/961204507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 msgr2=0x55bdf4502200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf4502200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 -- 172.21.15.132:0/961204507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 msgr2=0x55bdf44f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 -- 172.21.15.132:0/961204507 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf4225620 con 0x55bdf44ea5e0 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff009b58640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00a359640 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feffc042a30 tx=0x7feffc042e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a106a545f28108f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7fefef7fe640 1 -- 172.21.15.132:0/961204507 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc065070 con 0x55bdf44ea5e0 2024-03-18T23:10:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7fefef7fe640 1 -- 172.21.15.132:0/961204507 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc053070 con 0x55bdf44ea5e0 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7fefef7fe640 1 -- 172.21.15.132:0/961204507 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc039070 con 0x55bdf44ea5e0 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 -- 172.21.15.132:0/961204507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 msgr2=0x55bdf44e9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feffc042a30 tx=0x7feffc042e80 comp rx=0 tx=0).stop 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 -- 172.21.15.132:0/961204507 shutdown_connections 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e9a20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44e1810 0x55bdf4502200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.129+0000 7ff00e99c740 1 --2- 172.21.15.132:0/961204507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- 172.21.15.132:0/961204507 >> 172.21.15.132:0/961204507 conn(0x55bdf44db810 msgr2=0x55bdf44dbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- 172.21.15.132:0/961204507 shutdown_connections 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- 172.21.15.132:0/961204507 wait complete. 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- start start 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 Processor -- start 2024-03-18T23:10:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b2040 con 0x55bdf44e1810 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b34b0 con 0x55bdf44e0ef0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041b2710 con 0x55bdf44ea5e0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff00e99c740 10 client.0 _ll_register_callbacks cb 0x7ff000005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff009357640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff009357640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41990/0 (socket says 172.21.15.132:41990) 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff009357640 1 -- 172.21.15.132:0/4118478117 learned_addr learned my addr 172.21.15.132:0/4118478117 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.130+0000 7ff009b58640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00a359640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 -- 172.21.15.132:0/4118478117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44ea5e0 msgr2=0x55bdf44e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 -- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 msgr2=0x55bdf44f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 -- 172.21.15.132:0/4118478117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf44e4a30 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009b58640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff009357640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feff409b4c0 tx=0x7feff406b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26aab55d8827f07d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff409b7c0 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feff4097e30 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feffc053070 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feff409f910 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 10 client.15450 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdf4560890 con 0x55bdf44e1810 2024-03-18T23:10:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feffc053070 con 0x55bdf44e1810 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7feff40cf030 con 0x55bdf44e1810 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 10 client.15450 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.131+0000 7ff00e99c740 20 client.15450 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf4561040 0x55bdf4563460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7ff00e99c740 4 client.15450 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdf444c470 con 0x55bdf4561040 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7ff009b58640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf4561040 0x55bdf4563460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7ff009b58640 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf4561040 0x55bdf4563460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72435d5c0040ce0b server_cookie=c9e6dc3eff7346cd in_seq=0 out_seq=0 2024-03-18T23:10:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.132+0000 7fefed7fa640 10 client.15450 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7fefed7fa640 1 -- 172.21.15.132:0/4118478117 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdf444c470 con 0x55bdf4561040 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7fefed7fa640 10 client.15450 handle_command_reply: tid=0 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 client.15450 shutdown 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 msgr2=0x55bdf44e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feff409b4c0 tx=0x7feff406b290 comp rx=0 tx=0).stop 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 shutdown_connections 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55bdf4561040 0x55bdf4563460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bdf44ea5e0 0x55bdf44e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdf44e1810 0x55bdf44e3cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 --2- 172.21.15.132:0/4118478117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bdf44e0ef0 0x55bdf44f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7fefed7fa640 0 client.15450 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 >> 172.21.15.132:0/4118478117 conn(0x55bdf44db810 msgr2=0x55bdf44da220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 shutdown_connections 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 1 -- 172.21.15.132:0/4118478117 wait complete. 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.142+0000 7ff00e99c740 20 client.15450 trim_cache size 0 max 16384 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:10:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:10:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 msgr2=0x7feff00a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 0x7feff00a4ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feffc037040 tx=0x7feffc005c50 comp rx=0 tx=0).stop 2024-03-18T23:10:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 msgr2=0x7ff00419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feff400ea60 tx=0x7feff400ef20 comp rx=0 tx=0).stop 2024-03-18T23:10:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 shutdown_connections 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feff00a2b30 0x7feff00a4ff0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0041067e0 0x7ff0041a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff004104230 0x7ff00419ea10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 --2- 172.21.15.132:0/505116015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0040693e0 0x7ff00419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 >> 172.21.15.132:0/505116015 conn(0x7ff004077570 msgr2=0x7ff0040ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 shutdown_connections 2024-03-18T23:10:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:10.143+0000 7ff00bdbc640 1 -- 172.21.15.132:0/505116015 wait complete. 2024-03-18T23:10:11.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:11 smithi144 ceph-mon[66781]: pgmap v31: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 32 op/s 2024-03-18T23:10:11.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:11 smithi132 ceph-mon[32020]: pgmap v31: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 32 op/s 2024-03-18T23:10:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:11 smithi196 ceph-mon[67089]: pgmap v31: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 32 op/s 2024-03-18T23:10:13.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:13 smithi144 ceph-mon[66781]: pgmap v32: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-03-18T23:10:13.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:13 smithi132 ceph-mon[32020]: pgmap v32: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-03-18T23:10:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:13 smithi196 ceph-mon[67089]: pgmap v32: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-03-18T23:10:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:15 smithi144 ceph-mon[66781]: pgmap v33: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-18T23:10:15.616 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:15 smithi132 ceph-mon[32020]: pgmap v33: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-18T23:10:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:15 smithi196 ceph-mon[67089]: pgmap v33: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-03-18T23:10:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:17 smithi144 ceph-mon[66781]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:10:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:17 smithi132 ceph-mon[32020]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:10:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:17 smithi196 ceph-mon[67089]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:10:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:19 smithi144 ceph-mon[66781]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:19.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:19 smithi132 ceph-mon[32020]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:19 smithi196 ceph-mon[67089]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:20.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:20 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:20.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:20 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:20.624 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:20 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:10:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:21 smithi196 ceph-mon[67089]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:21 smithi144 ceph-mon[66781]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:10:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:10:21.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:10:21.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:21 smithi132 ceph-mon[32020]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 -- 172.21.15.132:0/3566793131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 msgr2=0x7f305010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 --2- 172.21.15.132:0/3566793131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f305010d550 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3048009ca0 tx=0x7f304802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 -- 172.21.15.132:0/3566793131 shutdown_connections 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 --2- 172.21.15.132:0/3566793131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f305010d550 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 --2- 172.21.15.132:0/3566793131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 0x7f3050100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 --2- 172.21.15.132:0/3566793131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 0x7f3050105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 -- 172.21.15.132:0/3566793131 >> 172.21.15.132:0/3566793131 conn(0x7f3050077570 msgr2=0x7f3050077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 -- 172.21.15.132:0/3566793131 shutdown_connections 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.236+0000 7f30552c6640 1 -- 172.21.15.132:0/3566793131 wait complete. 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 Processor -- start 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 -- start start 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 0x7f3050196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 0x7f305019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3050110be0 con 0x7f3050101330 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3050110a60 con 0x7f30501056c0 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f30552c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3050110d60 con 0x7f3050100970 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f304e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f304e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54816/0 (socket says 172.21.15.132:54816) 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f304e7fc640 1 -- 172.21.15.132:0/1327588411 learned_addr learned my addr 172.21.15.132:0/1327588411 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f304effd640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 0x7f3050196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.237+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 0x7f305019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 -- 172.21.15.132:0/1327588411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 msgr2=0x7f3050196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 0x7f3050196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 -- 172.21.15.132:0/1327588411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 msgr2=0x7f305019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 0x7f305019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 -- 172.21.15.132:0/1327588411 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305019bf10 con 0x7f3050101330 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f304e7fc640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f304000ea60 tx=0x7f304000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f3037fff640 1 -- 172.21.15.132:0/1327588411 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304000cde0 con 0x7f3050101330 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305019c200 con 0x7f3050101330 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f3037fff640 1 -- 172.21.15.132:0/1327588411 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30400035f0 con 0x7f3050101330 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f3037fff640 1 -- 172.21.15.132:0/1327588411 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040010670 con 0x7f3050101330 2024-03-18T23:10:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.238+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30501a3a90 con 0x7f3050101330 2024-03-18T23:10:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3037fff640 1 -- 172.21.15.132:0/1327588411 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 137524+0+0 (secure 0 0 0) 0x7f30400037a0 con 0x7f3050101330 2024-03-18T23:10:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 Processor -- start 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 -- start start 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb5b5e0 0x5566cdb5d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5566cdab5be0 con 0x5566cdb52810 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3037fff640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 0x7f30380a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5566cda5e960 con 0x5566cdb5b5e0 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f3057f93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566cdab1c80 con 0x5566cdb51ef0 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 0x7f30380a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb5b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54818/0 (socket says 172.21.15.132:54818) 2024-03-18T23:10:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304effd640 1 -- 172.21.15.132:0/2286461314 learned_addr learned my addr 172.21.15.132:0/2286461314 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.241+0000 7f304e7fc640 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb5b5e0 0x5566cdb5d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f3037fff640 1 -- 172.21.15.132:0/1327588411 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f30400cf3e0 con 0x7f3050101330 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 -- 172.21.15.132:0/2286461314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb5b5e0 msgr2=0x5566cdb5d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb5b5e0 0x5566cdb5d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 -- 172.21.15.132:0/2286461314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 msgr2=0x5566cdb5b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb5b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 -- 172.21.15.132:0/2286461314 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566cda55d20 con 0x5566cdb51ef0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 0x7f30380a4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3048032530 tx=0x7f3048048000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f304f7fe640 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb522d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f304802c700 tx=0x7f30480328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ffd605be7b6173 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f30357fa640 1 -- 172.21.15.132:0/2286461314 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048039040 con 0x5566cdb51ef0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f30357fa640 1 -- 172.21.15.132:0/2286461314 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048051070 con 0x5566cdb51ef0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.242+0000 7f30357fa640 1 -- 172.21.15.132:0/2286461314 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480346a0 con 0x5566cdb51ef0 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- 172.21.15.132:0/2286461314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 msgr2=0x5566cdb522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb522d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f304802c700 tx=0x7f30480328a0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- 172.21.15.132:0/2286461314 shutdown_connections 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb5b5e0 0x5566cdb5d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 --2- 172.21.15.132:0/2286461314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb522d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- 172.21.15.132:0/2286461314 >> 172.21.15.132:0/2286461314 conn(0x5566cdb4c810 msgr2=0x5566cdb4cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- 172.21.15.132:0/2286461314 shutdown_connections 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- 172.21.15.132:0/2286461314 wait complete. 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 -- start start 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.243+0000 7f3057f93740 1 Processor -- start 2024-03-18T23:10:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb52810 0x5566cdb58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 0x5566cdb58950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5566cdb5e1b0 con 0x5566cdb51ef0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5566cda71db0 con 0x5566cdb52810 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566cdb5e330 con 0x5566cdb5b5e0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 10 client.0 _ll_register_callbacks cb 0x5566cda76e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54822/0 (socket says 172.21.15.132:54822) 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 -- 172.21.15.132:0/682507047 learned_addr learned my addr 172.21.15.132:0/682507047 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 0x5566cdb58950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb52810 0x5566cdb58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 -- 172.21.15.132:0/682507047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 msgr2=0x5566cdb58950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 0x5566cdb58950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 -- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb52810 msgr2=0x5566cdb58340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb52810 0x5566cdb58340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 -- 172.21.15.132:0/682507047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566cda70c60 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304f7fe640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 0x5566cdb58950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f304effd640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3044012840 tx=0x7f3044012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca773d3c0ad0c783 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3032ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044004050 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f30440041f0 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.244+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048051070 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f30440043f0 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 10 client.15462 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5566cd7ca210 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048051070 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f3044003b70 con 0x5566cdb51ef0 2024-03-18T23:10:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 10 client.15462 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 20 client.15462 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb6b490 0x5566cdb6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 4 client.15462 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566cdac3120 con 0x5566cdb6b490 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.245+0000 7f304e7fc640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb6b490 0x5566cdb6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.246+0000 7f3032ff5640 10 client.15462 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.246+0000 7f304e7fc640 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb6b490 0x5566cdb6d8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffff08be0135db4b server_cookie=8dd943872e5df711 in_seq=0 out_seq=0 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.246+0000 7f3032ff5640 1 -- 172.21.15.132:0/682507047 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566cdac3120 con 0x5566cdb6b490 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.246+0000 7f3032ff5640 10 client.15462 handle_command_reply: tid=0 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.246+0000 7f3057f93740 1 client.15462 shutdown 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 msgr2=0x5566cdb5afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3044012840 tx=0x7f3044012d00 comp rx=0 tx=0).stop 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 shutdown_connections 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb6b490 0x5566cdb6d8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb5b5e0 0x5566cdb58950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb52810 0x5566cdb58340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3032ff5640 0 client.15462 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 --2- 172.21.15.132:0/682507047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb5afb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 >> 172.21.15.132:0/682507047 conn(0x5566cdb4c810 msgr2=0x5566cdb4bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 shutdown_connections 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 1 -- 172.21.15.132:0/682507047 wait complete. 2024-03-18T23:10:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.247+0000 7f3057f93740 20 client.15462 trim_cache size 0 max 16384 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.261+0000 7f3057f93740 1 Processor -- start 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.261+0000 7f3057f93740 1 -- start start 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.261+0000 7f3057f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3057f93740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3057f93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 0x5566cdb731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3057f93740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f30501aea90 con 0x5566cdb51ef0 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3057f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30501ac360 con 0x5566cdb4c810 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3057f93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30501aa1b0 con 0x5566cdb52810 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:59716/0 (socket says 172.21.15.132:59716) 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54828/0 (socket says 172.21.15.132:54828) 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304effd640 1 -- 172.21.15.132:0/3190701606 learned_addr learned my addr 172.21.15.132:0/3190701606 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304f7fe640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 0x5566cdb731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 -- 172.21.15.132:0/3190701606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 msgr2=0x5566cdb731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 0x5566cdb731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 -- 172.21.15.132:0/3190701606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 msgr2=0x5566cdb66f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 -- 172.21.15.132:0/3190701606 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566cd896620 con 0x5566cdb51ef0 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304effd640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304f7fe640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 0x5566cdb731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f304e7fc640 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30400cc9e0 tx=0x7f30400148c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1377029e06bdf14a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3034ff9640 1 -- 172.21.15.132:0/3190701606 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304009bf20 con 0x5566cdb51ef0 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3034ff9640 1 -- 172.21.15.132:0/3190701606 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304009b820 con 0x5566cdb51ef0 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.262+0000 7f3034ff9640 1 -- 172.21.15.132:0/3190701606 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400860d0 con 0x5566cdb51ef0 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- 172.21.15.132:0/3190701606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 msgr2=0x5566cdb63770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30400cc9e0 tx=0x7f30400148c0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- 172.21.15.132:0/3190701606 shutdown_connections 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb52810 0x5566cdb731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb51ef0 0x5566cdb63770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- 172.21.15.132:0/3190701606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb66f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- 172.21.15.132:0/3190701606 >> 172.21.15.132:0/3190701606 conn(0x5566cdb4be40 msgr2=0x5566cdb4cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- 172.21.15.132:0/3190701606 shutdown_connections 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- 172.21.15.132:0/3190701606 wait complete. 2024-03-18T23:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- start start 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 Processor -- start 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f30501aa820 con 0x5566cdb52810 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30501abc90 con 0x5566cdb4c810 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30501aaef0 con 0x5566cdb51ef0 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.263+0000 7f3057f93740 10 client.0 _ll_register_callbacks cb 0x7f3044009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54834/0 (socket says 172.21.15.132:54834) 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 -- 172.21.15.132:0/1542282863 learned_addr learned my addr 172.21.15.132:0/1542282863 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304e7fc640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 -- 172.21.15.132:0/1542282863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 msgr2=0x5566cdb55e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb55e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 -- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 msgr2=0x5566cdb558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 -- 172.21.15.132:0/1542282863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566cdb54d40 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3032ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f304f7fe640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f304802c130 tx=0x7f3048034b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1790d9b8ef48f44d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304805e680 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304009b820 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f304805e820 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f304805ea20 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3057f93740 10 client.15468 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5566cdbd4280 con 0x5566cdb52810 2024-03-18T23:10:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.264+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304009b820 con 0x5566cdb52810 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f304804b070 con 0x5566cdb52810 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 10 client.15468 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 20 client.15468 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb5b5e0 0x5566cdbd6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 4 client.15468 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566cdabd470 con 0x5566cdb5b5e0 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f304effd640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb5b5e0 0x5566cdbd6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f304effd640 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb5b5e0 0x5566cdbd6650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5f083e88b1106e server_cookie=5ab1523cf4586042 in_seq=0 out_seq=0 2024-03-18T23:10:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.265+0000 7f3032ff5640 10 client.15468 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3032ff5640 1 -- 172.21.15.132:0/1542282863 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566cdabd470 con 0x5566cdb5b5e0 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3032ff5640 10 client.15468 handle_command_reply: tid=0 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 client.15468 shutdown 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 msgr2=0x5566cdb545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f304802c130 tx=0x7f3048034b00 comp rx=0 tx=0).stop 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 shutdown_connections 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566cdb5b5e0 0x5566cdbd6650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566cdb52810 0x5566cdb545d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566cdb51ef0 0x5566cdb55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 --2- 172.21.15.132:0/1542282863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566cdb4c810 0x5566cdb558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3032ff5640 0 client.15468 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 >> 172.21.15.132:0/1542282863 conn(0x5566cdb4be40 msgr2=0x5566cdb4daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 shutdown_connections 2024-03-18T23:10:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.274+0000 7f3057f93740 1 -- 172.21.15.132:0/1542282863 wait complete. 2024-03-18T23:10:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f3057f93740 20 client.15468 trim_cache size 0 max 16384 2024-03-18T23:10:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:10:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:10:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:10:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 msgr2=0x7f30380a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 0x7f30380a4ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3048032530 tx=0x7f3048048000 comp rx=0 tx=0).stop 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 msgr2=0x7f3050197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f304000ea60 tx=0x7f304000ef20 comp rx=0 tx=0).stop 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 shutdown_connections 2024-03-18T23:10:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f30380a2b30 0x7f30380a4ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30501056c0 0x7f305019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3050101330 0x7f3050197460 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 --2- 172.21.15.132:0/1327588411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3050100970 0x7f3050196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 >> 172.21.15.132:0/1327588411 conn(0x7f3050077570 msgr2=0x7f30500ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 shutdown_connections 2024-03-18T23:10:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:23.275+0000 7f30552c6640 1 -- 172.21.15.132:0/1327588411 wait complete. 2024-03-18T23:10:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:23 smithi144 ceph-mon[66781]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-03-18T23:10:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:23 smithi132 ceph-mon[32020]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-03-18T23:10:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:23 smithi196 ceph-mon[67089]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-03-18T23:10:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:25 smithi144 ceph-mon[66781]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:10:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:25 smithi132 ceph-mon[32020]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:10:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:25 smithi196 ceph-mon[67089]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:10:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:27 smithi144 ceph-mon[66781]: pgmap v39: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:27.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:27 smithi132 ceph-mon[32020]: pgmap v39: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:27 smithi196 ceph-mon[67089]: pgmap v39: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:29.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:29 smithi144 ceph-mon[66781]: pgmap v40: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2024-03-18T23:10:29.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:29 smithi132 ceph-mon[32020]: pgmap v40: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2024-03-18T23:10:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:29 smithi196 ceph-mon[67089]: pgmap v40: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.6 KiB/s wr, 12 op/s 2024-03-18T23:10:31.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:31 smithi144 ceph-mon[66781]: pgmap v41: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:31.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:31 smithi132 ceph-mon[32020]: pgmap v41: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:31 smithi196 ceph-mon[67089]: pgmap v41: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-03-18T23:10:33.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:33 smithi144 ceph-mon[66781]: pgmap v42: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:33.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:33 smithi132 ceph-mon[32020]: pgmap v42: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:33 smithi196 ceph-mon[67089]: pgmap v42: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:35.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:35 smithi144 ceph-mon[66781]: pgmap v43: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:35.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:35 smithi132 ceph-mon[32020]: pgmap v43: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:35 smithi196 ceph-mon[67089]: pgmap v43: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:10:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 -- 172.21.15.132:0/518071431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb00ffcc0 msgr2=0x7fccb0100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 --2- 172.21.15.132:0/518071431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb00ffcc0 0x7fccb0100140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009930 tx=0x7fcca402bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 -- 172.21.15.132:0/518071431 shutdown_connections 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 --2- 172.21.15.132:0/518071431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb0100680 0x7fccb010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 --2- 172.21.15.132:0/518071431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb00ffcc0 0x7fccb0100140 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 --2- 172.21.15.132:0/518071431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb01077e0 0x7fccb0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 -- 172.21.15.132:0/518071431 >> 172.21.15.132:0/518071431 conn(0x7fccb0076c40 msgr2=0x7fccb00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 -- 172.21.15.132:0/518071431 shutdown_connections 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.371+0000 7fccb56b2640 1 -- 172.21.15.132:0/518071431 wait complete. 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 Processor -- start 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 -- start start 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb00ffcc0 0x7fccb0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb0100680 0x7fccb01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0111bf0 con 0x7fccb01077e0 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0111a70 con 0x7fccb00ffcc0 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.372+0000 7fccb56b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0111d70 con 0x7fccb0100680 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb00ffcc0 0x7fccb0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36704/0 (socket says 172.21.15.132:36704) 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 -- 172.21.15.132:0/1686186133 learned_addr learned my addr 172.21.15.132:0/1686186133 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccae7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb0100680 0x7fccb01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 -- 172.21.15.132:0/1686186133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb0100680 msgr2=0x7fccb01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb0100680 0x7fccb01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 -- 172.21.15.132:0/1686186133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb00ffcc0 msgr2=0x7fccb0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb00ffcc0 0x7fccb0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 -- 172.21.15.132:0/1686186133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb019d060 con 0x7fccb01077e0 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcca800e9b0 tx=0x7fcca800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fcc97fff640 1 -- 172.21.15.132:0/1686186133 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca800cd80 con 0x7fccb01077e0 2024-03-18T23:10:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb019d350 con 0x7fccb01077e0 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fcc97fff640 1 -- 172.21.15.132:0/1686186133 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca80035f0 con 0x7fccb01077e0 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fcc97fff640 1 -- 172.21.15.132:0/1686186133 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8010670 con 0x7fccb01077e0 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.373+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01a4be0 con 0x7fccb01077e0 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.375+0000 7fcc97fff640 1 -- 172.21.15.132:0/1686186133 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) ==== 137524+0+0 (secure 0 0 0) 0x7fcca80037a0 con 0x7fccb01077e0 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 Processor -- start 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 -- start start 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fcc97fff640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 0x7fcc980a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a9a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a9c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55c3749f4be0 con 0x55c374a90ef0 2024-03-18T23:10:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c37499d960 con 0x55c374a91810 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccb837c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c3749f0c80 con 0x55c374a9a5e0 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccae7fc640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 0x7fcc980a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36708/0 (socket says 172.21.15.132:36708) 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccaeffd640 1 -- 172.21.15.132:0/4137984873 learned_addr learned my addr 172.21.15.132:0/4137984873 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fcc97fff640 1 -- 172.21.15.132:0/1686186133 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcca80cf3c0 con 0x7fccb01077e0 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a9c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.376+0000 7fccae7fc640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a9a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 -- 172.21.15.132:0/4137984873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 msgr2=0x55c374a9c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a9c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 -- 172.21.15.132:0/4137984873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 msgr2=0x55c374a9a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a9a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 -- 172.21.15.132:0/4137984873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c374994d20 con 0x55c374a90ef0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a9c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccae7fc640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 0x7fcc980a4ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005ab0 tx=0x7fcca4005e20 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fccaeffd640 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009fd0 tx=0x7fcca00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a220920fc226e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fcc957fa640 1 -- 172.21.15.132:0/4137984873 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004d20 con 0x55c374a90ef0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fcc957fa640 1 -- 172.21.15.132:0/4137984873 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca0004050 con 0x55c374a90ef0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.377+0000 7fcc957fa640 1 -- 172.21.15.132:0/4137984873 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0005240 con 0x55c374a90ef0 2024-03-18T23:10:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- 172.21.15.132:0/4137984873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 msgr2=0x55c374a912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcca0009fd0 tx=0x7fcca00118d0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- 172.21.15.132:0/4137984873 shutdown_connections 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a9c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a9a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- 172.21.15.132:0/4137984873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a912d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- 172.21.15.132:0/4137984873 >> 172.21.15.132:0/4137984873 conn(0x55c374a8b810 msgr2=0x55c374a8bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- 172.21.15.132:0/4137984873 shutdown_connections 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- 172.21.15.132:0/4137984873 wait complete. 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- start start 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 Processor -- start 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55c374a9d210 con 0x55c374a90ef0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c3749b0db0 con 0x55c374a91810 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.378+0000 7fccb837c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c374a9d390 con 0x55c374a9a5e0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccb837c740 10 client.0 _ll_register_callbacks cb 0x55c3749b5e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36714/0 (socket says 172.21.15.132:36714) 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 -- 172.21.15.132:0/336072769 learned_addr learned my addr 172.21.15.132:0/336072769 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccae7fc640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 -- 172.21.15.132:0/336072769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 msgr2=0x55c374a977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 -- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 msgr2=0x55c374a97270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a97270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 -- 172.21.15.132:0/336072769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3749afc60 con 0x55c374a90ef0 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccae7fc640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a97270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccaeffd640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcca0011dd0 tx=0x7fcca0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4b4c13b3b2abf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fcc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0029020 con 0x55c374a90ef0 2024-03-18T23:10:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcca0004c40 con 0x55c374a90ef0 2024-03-18T23:10:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca0004050 con 0x55c374a90ef0 2024-03-18T23:10:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcca0034050 con 0x55c374a90ef0 2024-03-18T23:10:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccb837c740 10 client.15480 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c374709210 con 0x55c374a90ef0 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.379+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca0004050 con 0x55c374a90ef0 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fcca000f4e0 con 0x55c374a90ef0 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 10 client.15480 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 20 client.15480 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374aaa370 0x55c374aac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 4 client.15480 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c374a02120 con 0x55c374aaa370 2024-03-18T23:10:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccae7fc640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374aaa370 0x55c374aac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fcc92ff5640 10 client.15480 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.380+0000 7fccae7fc640 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374aaa370 0x55c374aac760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d040560a9ea3a4a3 server_cookie=33212d10c1a771a9 in_seq=0 out_seq=0 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fcc92ff5640 1 -- 172.21.15.132:0/336072769 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c374a02120 con 0x55c374aaa370 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fcc92ff5640 10 client.15480 handle_command_reply: tid=0 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 client.15480 shutdown 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 msgr2=0x55c374a99ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcca0011dd0 tx=0x7fcca0009c80 comp rx=0 tx=0).stop 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 shutdown_connections 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374aaa370 0x55c374aac760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374a977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374a97270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fcc92ff5640 0 client.15480 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 --2- 172.21.15.132:0/336072769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a99ee0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.381+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 >> 172.21.15.132:0/336072769 conn(0x55c374a8b810 msgr2=0x55c374a8aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.382+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 shutdown_connections 2024-03-18T23:10:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.382+0000 7fccb837c740 1 -- 172.21.15.132:0/336072769 wait complete. 2024-03-18T23:10:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.382+0000 7fccb837c740 20 client.15480 trim_cache size 0 max 16384 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 Processor -- start 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 -- start start 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374ab1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374aa25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01afcd0 con 0x55c374a90ef0 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01ad5a0 con 0x55c374a91810 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccb837c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01ab3f0 con 0x55c374a9a5e0 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.396+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36722/0 (socket says 172.21.15.132:36722) 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 -- 172.21.15.132:0/3830281951 learned_addr learned my addr 172.21.15.132:0/3830281951 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374aa25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccae7fc640 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374ab1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 -- 172.21.15.132:0/3830281951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 msgr2=0x55c374aa25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374aa25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 -- 172.21.15.132:0/3830281951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 msgr2=0x55c374ab1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374ab1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 -- 172.21.15.132:0/3830281951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3747d5620 con 0x55c374a90ef0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fccaeffd640 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcca00039f0 tx=0x7fcca0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d3ec574d358b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fcc94ff9640 1 -- 172.21.15.132:0/3830281951 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0029040 con 0x55c374a90ef0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fcc94ff9640 1 -- 172.21.15.132:0/3830281951 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca000f4e0 con 0x55c374a90ef0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.397+0000 7fcc94ff9640 1 -- 172.21.15.132:0/3830281951 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0005030 con 0x55c374a90ef0 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- 172.21.15.132:0/3830281951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 msgr2=0x55c374a8a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcca00039f0 tx=0x7fcca0004760 comp rx=0 tx=0).stop 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- 172.21.15.132:0/3830281951 shutdown_connections 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a9a5e0 0x55c374aa25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a91810 0x55c374ab1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- 172.21.15.132:0/3830281951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374a8a930 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- 172.21.15.132:0/3830281951 >> 172.21.15.132:0/3830281951 conn(0x55c374a8b810 msgr2=0x55c374a8a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- 172.21.15.132:0/3830281951 shutdown_connections 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- 172.21.15.132:0/3830281951 wait complete. 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- start start 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 Processor -- start 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a91810 0x55c374a9d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a9a5e0 0x55c374a9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01aba60 con 0x55c374a90ef0 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01aced0 con 0x55c374a9a5e0 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01ac130 con 0x55c374a91810 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccb837c740 10 client.0 _ll_register_callbacks cb 0x7fcca00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccaeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36734/0 (socket says 172.21.15.132:36734) 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccaeffd640 1 -- 172.21.15.132:0/749677182 learned_addr learned my addr 172.21.15.132:0/749677182 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccae7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a91810 0x55c374a9d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.398+0000 7fccaf7fe640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a9a5e0 0x55c374a9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 -- 172.21.15.132:0/749677182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a91810 msgr2=0x55c374a9d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a91810 0x55c374a9d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 -- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a9a5e0 msgr2=0x55c374a9deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a9a5e0 0x55c374a9deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 -- 172.21.15.132:0/749677182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c374a9e3f0 con 0x55c374a90ef0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccaeffd640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcca000f980 tx=0x7fcca0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d746b9436ecafc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0029040 con 0x55c374a90ef0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcca0003f10 con 0x55c374a90ef0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca000f4e0 con 0x55c374a90ef0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcca0049050 con 0x55c374a90ef0 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 10 client.15486 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c374b102a0 con 0x55c374a90ef0 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcca000f4e0 con 0x55c374a90ef0 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fcca00053f0 con 0x55c374a90ef0 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 10 client.15486 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 20 client.15486 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374b10900 0x55c374b12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 4 client.15486 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.399+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3749fc470 con 0x55c374b10900 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.400+0000 7fccae7fc640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374b10900 0x55c374b12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.400+0000 7fccae7fc640 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374b10900 0x55c374b12cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d0bf65ec4a9697 server_cookie=1abfc9a6dfa11c5d in_seq=0 out_seq=0 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.400+0000 7fcc92ff5640 10 client.15486 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.407+0000 7fcc92ff5640 1 -- 172.21.15.132:0/749677182 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3749fc470 con 0x55c374b10900 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.407+0000 7fcc92ff5640 10 client.15486 handle_command_reply: tid=0 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.407+0000 7fccb837c740 1 client.15486 shutdown 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 msgr2=0x55c374aa25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcca000f980 tx=0x7fcca0011f10 comp rx=0 tx=0).stop 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 shutdown_connections 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55c374b10900 0x55c374b12cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c374a9a5e0 0x55c374a9deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c374a91810 0x55c374a9d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fcc92ff5640 0 client.15486 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 --2- 172.21.15.132:0/749677182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c374a90ef0 0x55c374aa25b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 >> 172.21.15.132:0/749677182 conn(0x55c374a8b810 msgr2=0x55c374a8aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 shutdown_connections 2024-03-18T23:10:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 1 -- 172.21.15.132:0/749677182 wait complete. 2024-03-18T23:10:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb837c740 20 client.15486 trim_cache size 0 max 16384 2024-03-18T23:10:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:10:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:10:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:10:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:10:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 msgr2=0x7fcc980a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 0x7fcc980a4ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcca4005ab0 tx=0x7fcca4005e20 comp rx=0 tx=0).stop 2024-03-18T23:10:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 msgr2=0x7fccb019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcca800e9b0 tx=0x7fcca800ee70 comp rx=0 tx=0).stop 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 shutdown_connections 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcc980a2b30 0x7fcc980a4ff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb01077e0 0x7fccb019c930 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccb0100680 0x7fccb01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 --2- 172.21.15.132:0/1686186133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb00ffcc0 0x7fccb0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 >> 172.21.15.132:0/1686186133 conn(0x7fccb0076c40 msgr2=0x7fccb01045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.408+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 shutdown_connections 2024-03-18T23:10:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:36.409+0000 7fccb56b2640 1 -- 172.21.15.132:0/1686186133 wait complete. 2024-03-18T23:10:37.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:37 smithi144 ceph-mon[66781]: pgmap v44: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:37.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:37 smithi132 ceph-mon[32020]: pgmap v44: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:37 smithi196 ceph-mon[67089]: pgmap v44: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:39 smithi144 ceph-mon[66781]: pgmap v45: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-18T23:10:39.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:39 smithi132 ceph-mon[32020]: pgmap v45: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-18T23:10:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:39 smithi196 ceph-mon[67089]: pgmap v45: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-18T23:10:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:41 smithi144 ceph-mon[66781]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:41 smithi132 ceph-mon[32020]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:41.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:41 smithi196 ceph-mon[67089]: pgmap v46: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:10:43.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:43 smithi144 ceph-mon[66781]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:43.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:43 smithi132 ceph-mon[32020]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:43.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:43 smithi196 ceph-mon[67089]: pgmap v47: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:45.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:45 smithi144 ceph-mon[66781]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:45.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:45 smithi132 ceph-mon[32020]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:45 smithi196 ceph-mon[67089]: pgmap v48: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:47.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:47 smithi132 ceph-mon[32020]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T23:10:47.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:47 smithi132 ceph-mon[32020]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:47 smithi196 ceph-mon[67089]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T23:10:47.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:47 smithi196 ceph-mon[67089]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:47 smithi144 ceph-mon[66781]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T23:10:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:47 smithi144 ceph-mon[66781]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 -- 172.21.15.132:0/2227393787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c1077f0 msgr2=0x7f540c107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 --2- 172.21.15.132:0/2227393787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c1077f0 0x7f540c107bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009900 tx=0x7f53fc02bc30 comp rx=0 tx=0).stop 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 -- 172.21.15.132:0/2227393787 shutdown_connections 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 --2- 172.21.15.132:0/2227393787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c10e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 --2- 172.21.15.132:0/2227393787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c06a020 0x7f540c06a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 --2- 172.21.15.132:0/2227393787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c1077f0 0x7f540c107bd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 -- 172.21.15.132:0/2227393787 >> 172.21.15.132:0/2227393787 conn(0x7f540c0774f0 msgr2=0x7f540c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 -- 172.21.15.132:0/2227393787 shutdown_connections 2024-03-18T23:10:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.503+0000 7f5410c0b640 1 -- 172.21.15.132:0/2227393787 wait complete. 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.504+0000 7f5410c0b640 1 Processor -- start 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.504+0000 7f5410c0b640 1 -- start start 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 0x7f540c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 0x7f540c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111bc0 con 0x7f540c06a020 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111a40 con 0x7f540c06a9c0 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5410c0b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f540c111d40 con 0x7f540c1077f0 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f540a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 0x7f540c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 0x7f540c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 0x7f540c1014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:52244/0 (socket says 172.21.15.132:52244) 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f540a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 0x7f540c100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46110/0 (socket says 172.21.15.132:46110) 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f540ad9d640 1 -- 172.21.15.132:0/3275550359 learned_addr learned my addr 172.21.15.132:0/3275550359 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 -- 172.21.15.132:0/3275550359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 msgr2=0x7f540c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 0x7f540c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 -- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 msgr2=0x7f540c100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 0x7f540c100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 -- 172.21.15.132:0/3275550359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1a64e0 con 0x7f540c06a9c0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.505+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c100f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9f0 tx=0x7f53f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.506+0000 7f5403fff640 1 -- 172.21.15.132:0/3275550359 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400ccc0 con 0x7f540c06a9c0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.506+0000 7f5403fff640 1 -- 172.21.15.132:0/3275550359 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f4010910 con 0x7f540c06a9c0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.506+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a67d0 con 0x7f540c06a9c0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.506+0000 7f5403fff640 1 -- 172.21.15.132:0/3275550359 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4018960 con 0x7f540c06a9c0 2024-03-18T23:10:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.506+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a6d70 con 0x7f540c06a9c0 2024-03-18T23:10:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f5403fff640 1 -- 172.21.15.132:0/3275550359 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f53f4010ac0 con 0x7f540c06a9c0 2024-03-18T23:10:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f5403fff640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 0x7f53f80a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f54137fb740 1 Processor -- start 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f540a59c640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 0x7f53f80a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f5403fff640 1 -- 172.21.15.132:0/3275550359 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f53f40cf0b0 con 0x7f540c06a9c0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.508+0000 7f54137fb740 1 -- start start 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55ccde110be0 con 0x55ccde1b65e0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccde0b9960 con 0x55ccde1ad810 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f54137fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccde10cc80 con 0x55ccde1acef0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f540a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:47454/0 (socket says 172.21.15.132:47454) 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 -- 172.21.15.132:0/902298399 learned_addr learned my addr 172.21.15.132:0/902298399 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f540a59c640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 0x7f53f80a4f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006010 tx=0x7f53fc005dd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 -- 172.21.15.132:0/902298399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 msgr2=0x55ccde1ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 -- 172.21.15.132:0/902298399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 msgr2=0x55ccde1b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 -- 172.21.15.132:0/902298399 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccde0b0d20 con 0x55ccde1ad810 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.509+0000 7f5401d9b640 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53f409b590 tx=0x7f53f409b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8f9351c3e44e26 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f5400d99640 1 -- 172.21.15.132:0/902298399 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40a3f80 con 0x55ccde1ad810 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f5400d99640 1 -- 172.21.15.132:0/902298399 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f40ccb50 con 0x55ccde1ad810 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f5400d99640 1 -- 172.21.15.132:0/902298399 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f409af40 con 0x55ccde1ad810 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 -- 172.21.15.132:0/902298399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 msgr2=0x55ccde1b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53f409b590 tx=0x7f53f409b310 comp rx=0 tx=0).stop 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 -- 172.21.15.132:0/902298399 shutdown_connections 2024-03-18T23:10:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b60a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 --2- 172.21.15.132:0/902298399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.510+0000 7f54137fb740 1 -- 172.21.15.132:0/902298399 >> 172.21.15.132:0/902298399 conn(0x55ccde1a7810 msgr2=0x55ccde1a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- 172.21.15.132:0/902298399 shutdown_connections 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- 172.21.15.132:0/902298399 wait complete. 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- start start 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 Processor -- start 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55ccde1b91b0 con 0x55ccde1b65e0 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ccde0ccdb0 con 0x55ccde1ad810 2024-03-18T23:10:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccde1b9330 con 0x55ccde1acef0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f54137fb740 10 client.0 _ll_register_callbacks cb 0x55ccde0d1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f5401d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f540a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46142/0 (socket says 172.21.15.132:46142) 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.511+0000 7f540ad9d640 1 -- 172.21.15.132:0/3213094205 learned_addr learned my addr 172.21.15.132:0/3213094205 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 -- 172.21.15.132:0/3213094205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 msgr2=0x55ccde1b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 -- 172.21.15.132:0/3213094205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 msgr2=0x55ccde1b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 -- 172.21.15.132:0/3213094205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccde0cbc60 con 0x55ccde1ad810 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b33c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ccde1ac9a0 tx=0x7f53f400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310c7b04908f9269 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:10:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40a3f80 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f53f40afbc0 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f40ccb50 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f53f40afdc0 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f54137fb740 10 client.25343 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ccdde25210 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f40ccb50 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.512+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f53f409cca0 con 0x55ccde1ad810 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 10 client.25343 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 20 client.25343 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde1c6340 0x55ccde1c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 4 client.25343 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccde11e120 con 0x55ccde1c6340 2024-03-18T23:10:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f540a59c640 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde1c6340 0x55ccde1c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f540a59c640 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde1c6340 0x55ccde1c8730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfb03762958264d server_cookie=d30132841b358f15 in_seq=0 out_seq=0 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.513+0000 7f53f1ffb640 10 client.25343 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f53f1ffb640 1 -- 172.21.15.132:0/3213094205 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ccde11e120 con 0x55ccde1c6340 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f53f1ffb640 10 client.25343 handle_command_reply: tid=0 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 client.25343 shutdown 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 msgr2=0x55ccde1b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b33c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ccde1ac9a0 tx=0x7f53f400cf60 comp rx=0 tx=0).stop 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 shutdown_connections 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde1c6340 0x55ccde1c8730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1b33c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 --2- 172.21.15.132:0/3213094205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f53f1ffb640 0 client.25343 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 >> 172.21.15.132:0/3213094205 conn(0x55ccde1a7810 msgr2=0x55ccde1a6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 shutdown_connections 2024-03-18T23:10:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 1 -- 172.21.15.132:0/3213094205 wait complete. 2024-03-18T23:10:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.514+0000 7f54137fb740 20 client.25343 trim_cache size 0 max 16384 2024-03-18T23:10:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 Processor -- start 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 -- start start 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 0x55ccde1b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1b1f90 con 0x55ccde1acef0 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1af860 con 0x55ccde1ad810 2024-03-18T23:10:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f54137fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ad6b0 con 0x55ccde1b65e0 2024-03-18T23:10:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46156/0 (socket says 172.21.15.132:46156) 2024-03-18T23:10:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 -- 172.21.15.132:0/3651496584 learned_addr learned my addr 172.21.15.132:0/3651496584 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540ad9d640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 0x55ccde1b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 -- 172.21.15.132:0/3651496584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 msgr2=0x55ccde1b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 0x55ccde1b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 -- 172.21.15.132:0/3651496584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 msgr2=0x55ccde1cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540a59c640 1 -- 172.21.15.132:0/3651496584 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccddef1620 con 0x55ccde1acef0 2024-03-18T23:10:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f540ad9d640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 0x55ccde1b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:10:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.529+0000 7f5401d9b640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:10:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f540a59c640 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53fc02cfd0 tx=0x7f53fc044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d933ddc455542c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f53f3fff640 1 -- 172.21.15.132:0/3651496584 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc05b070 con 0x55ccde1acef0 2024-03-18T23:10:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f53f3fff640 1 -- 172.21.15.132:0/3651496584 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53fc044070 con 0x55ccde1acef0 2024-03-18T23:10:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f53f3fff640 1 -- 172.21.15.132:0/3651496584 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0052e0 con 0x55ccde1acef0 2024-03-18T23:10:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 -- 172.21.15.132:0/3651496584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 msgr2=0x55ccde1c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53fc02cfd0 tx=0x7f53fc044000 comp rx=0 tx=0).stop 2024-03-18T23:10:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 -- 172.21.15.132:0/3651496584 shutdown_connections 2024-03-18T23:10:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1b65e0 0x55ccde1b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 --2- 172.21.15.132:0/3651496584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1acef0 0x55ccde1c1f50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 -- 172.21.15.132:0/3651496584 >> 172.21.15.132:0/3651496584 conn(0x55ccde1a7810 msgr2=0x55ccde1a6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 -- 172.21.15.132:0/3651496584 shutdown_connections 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.530+0000 7f54137fb740 1 -- 172.21.15.132:0/3651496584 wait complete. 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 -- start start 2024-03-18T23:10:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 Processor -- start 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1add20 con 0x55ccde1b65e0 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1af190 con 0x55ccde1ad810 2024-03-18T23:10:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1ae3f0 con 0x55ccde1acef0 2024-03-18T23:10:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f54137fb740 10 client.0 _ll_register_callbacks cb 0x7f53fc042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:10:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46172/0 (socket says 172.21.15.132:46172) 2024-03-18T23:10:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 -- 172.21.15.132:0/2821259395 learned_addr learned my addr 172.21.15.132:0/2821259395 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:10:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540a59c640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f5401d9b640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 -- 172.21.15.132:0/2821259395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 msgr2=0x55ccde1be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 -- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 msgr2=0x55ccde1bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.531+0000 7f540ad9d640 1 -- 172.21.15.132:0/2821259395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccde1bc000 con 0x55ccde1b65e0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f540ad9d640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5404010c70 tx=0x7f540400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa1974d832313ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404021020 con 0x55ccde1b65e0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f5404003b70 con 0x55ccde1b65e0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc044070 con 0x55ccde1b65e0 2024-03-18T23:10:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5404003d70 con 0x55ccde1b65e0 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 10 client.15504 fetch_fsmap learned FSMap version 22 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ccde22d150 con 0x55ccde1b65e0 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53fc044070 con 0x55ccde1b65e0 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f540400e920 con 0x55ccde1b65e0 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 10 client.15504 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:10:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 20 client.15504 populate_metadata read hostname 'smithi132' 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde22da00 0x55ccde22fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 4 client.15504 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccde118470 con 0x55ccde22da00 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.532+0000 7f540a59c640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde22da00 0x55ccde22fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:10:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.533+0000 7f540a59c640 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde22da00 0x55ccde22fe20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd642c8841eb75f server_cookie=495690abff0f278b in_seq=0 out_seq=0 2024-03-18T23:10:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.533+0000 7f53f1ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f53f1ffb640 1 -- 172.21.15.132:0/2821259395 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccde118470 con 0x55ccde22da00 2024-03-18T23:10:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f53f1ffb640 10 client.15504 handle_command_reply: tid=0 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 client.15504 shutdown 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 msgr2=0x55ccde1bba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5404010c70 tx=0x7f540400f790 comp rx=0 tx=0).stop 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 shutdown_connections 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55ccde22da00 0x55ccde22fe20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ccde1b65e0 0x55ccde1bba90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ccde1ad810 0x55ccde1bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 --2- 172.21.15.132:0/2821259395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccde1acef0 0x55ccde1be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f53f1ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:10:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.539+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 >> 172.21.15.132:0/2821259395 conn(0x55ccde1a7810 msgr2=0x55ccde1a8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 shutdown_connections 2024-03-18T23:10:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f54137fb740 1 -- 172.21.15.132:0/2821259395 wait complete. 2024-03-18T23:10:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f54137fb740 20 client.15504 trim_cache size 0 max 16384 2024-03-18T23:10:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:10:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:10:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:10:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 msgr2=0x7f53f80a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 0x7f53f80a4f80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006010 tx=0x7f53fc005dd0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 msgr2=0x7f540c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c100f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9f0 tx=0x7f53f400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 shutdown_connections 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53f80a2ac0 0x7f53f80a4f80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f540c1077f0 0x7f540c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f540c06a9c0 0x7f540c100f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 --2- 172.21.15.132:0/3275550359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f540c06a020 0x7f540c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:10:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 >> 172.21.15.132:0/3275550359 conn(0x7f540c0774f0 msgr2=0x7f540c0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:10:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 shutdown_connections 2024-03-18T23:10:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:10:49.540+0000 7f5410c0b640 1 -- 172.21.15.132:0/3275550359 wait complete. 2024-03-18T23:10:49.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:49 smithi144 ceph-mon[66781]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:49.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:49 smithi132 ceph-mon[32020]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:49 smithi196 ceph-mon[67089]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-03-18T23:10:51.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:51 smithi144 ceph-mon[66781]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-03-18T23:10:51.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:51 smithi132 ceph-mon[32020]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-03-18T23:10:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:51 smithi196 ceph-mon[67089]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-03-18T23:10:53.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:53 smithi144 ceph-mon[66781]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:53 smithi132 ceph-mon[32020]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:53 smithi196 ceph-mon[67089]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:55.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:55 smithi144 ceph-mon[66781]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:55.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:55 smithi132 ceph-mon[32020]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:55 smithi196 ceph-mon[67089]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-03-18T23:10:57.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:57 smithi144 ceph-mon[66781]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:57.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:57 smithi132 ceph-mon[32020]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:57 smithi196 ceph-mon[67089]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-18T23:10:59.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:10:59 smithi144 ceph-mon[66781]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:59.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:10:59 smithi132 ceph-mon[32020]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:10:59 smithi196 ceph-mon[67089]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:1/600: open fd = 3 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:2/600: open fd = 3 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.845 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:3/600: open fd = 3 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:4/600: open fd = 3 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:5/600: open fd = 3 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:6/600: open fd = 3 2024-03-18T23:10:59.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:7/600: open fd = 3 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:8/600: open fd = 3 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:9/600: open fd = 3 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:10/600: open fd = 3 2024-03-18T23:10:59.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:11/600: open fd = 3 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:12/600: open fd = 3 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:13/600: open fd = 3 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:14/600: open fd = 3 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:15/600: open fd = 3 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:16/600: open fd = 3 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:17/600: open fd = 3 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.849 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:18/600: open fd = 3 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:19/600: open fd = 3 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:20/600: open fd = 3 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.850 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:21/600: open fd = 3 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:22/600: open fd = 3 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:23/600: open fd = 3 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:24/600: open fd = 3 2024-03-18T23:10:59.851 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:25/600: open fd = 3 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:26/600: open fd = 3 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:27/600: open fd = 3 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.852 INFO:tasks.workunit.client.0.smithi132.stdout:28/600: open fd = 3 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:29/600: open fd = 3 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:30/600: open fd = 3 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:31/600: open fd = 3 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.853 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:32/600: open fd = 3 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:33/600: open fd = 3 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:34/600: open fd = 3 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.854 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:35/600: open fd = 3 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:36/600: open fd = 3 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:37/600: open fd = 3 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:38/600: open fd = 3 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.855 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:39/600: open fd = 3 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:40/600: open fd = 3 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:41/600: open fd = 3 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.856 INFO:tasks.workunit.client.0.smithi132.stdout:42/600: open fd = 3 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:43/600: open fd = 3 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:44/600: open fd = 3 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:45/600: open fd = 3 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.857 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:46/600: open fd = 3 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:47/600: open fd = 3 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:48/600: open fd = 3 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.858 INFO:tasks.workunit.client.0.smithi132.stdout:49/600: open fd = 3 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:50/600: open fd = 3 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:51/600: open fd = 3 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:52/600: open fd = 3 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.859 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:53/600: open fd = 3 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:54/600: open fd = 3 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:55/600: open fd = 3 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:56/600: open fd = 3 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.860 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:57/600: open fd = 3 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:58/600: open fd = 3 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:59/600: open fd = 3 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:60/600: open fd = 3 2024-03-18T23:10:59.861 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:61/600: open fd = 3 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:62/600: open fd = 3 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:63/600: open fd = 3 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:10:59.862 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:11:01.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:01 smithi144 ceph-mon[66781]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:01.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:01 smithi132 ceph-mon[32020]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:01 smithi196 ceph-mon[67089]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 -- 172.21.15.132:0/3158748275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c1056a0 msgr2=0x7fb81c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 --2- 172.21.15.132:0/3158748275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c1056a0 0x7fb81c105a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb814009900 tx=0x7fb81402bc30 comp rx=0 tx=0).stop 2024-03-18T23:11:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 -- 172.21.15.132:0/3158748275 shutdown_connections 2024-03-18T23:11:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 --2- 172.21.15.132:0/3158748275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c101310 0x7fb81c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 --2- 172.21.15.132:0/3158748275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c100970 0x7fb81c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 --2- 172.21.15.132:0/3158748275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c1056a0 0x7fb81c105a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 -- 172.21.15.132:0/3158748275 >> 172.21.15.132:0/3158748275 conn(0x7fb81c0774f0 msgr2=0x7fb81c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 -- 172.21.15.132:0/3158748275 shutdown_connections 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.636+0000 7fb8230ac640 1 -- 172.21.15.132:0/3158748275 wait complete. 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 Processor -- start 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 -- start start 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c100970 0x7fb81c10e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 0x7fb81c10e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c111cd0 con 0x7fb81c1056a0 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c111b50 con 0x7fb81c101310 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb8230ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c111e50 con 0x7fb81c100970 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35570/0 (socket says 172.21.15.132:35570) 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 -- 172.21.15.132:0/1616457921 learned_addr learned my addr 172.21.15.132:0/1616457921 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb820e48640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c100970 0x7fb81c10e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 0x7fb81c10e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 -- 172.21.15.132:0/1616457921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c100970 msgr2=0x7fb81c10e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c100970 0x7fb81c10e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 -- 172.21.15.132:0/1616457921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 msgr2=0x7fb81c10e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.637+0000 7fb821649640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 0x7fb81c10e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb821649640 1 -- 172.21.15.132:0/1616457921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c1a6530 con 0x7fb81c1056a0 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 0x7fb81c10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb821649640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e970 tx=0x7fb80400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb819ffb640 1 -- 172.21.15.132:0/1616457921 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400ccc0 con 0x7fb81c1056a0 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c1a6820 con 0x7fb81c1056a0 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb819ffb640 1 -- 172.21.15.132:0/1616457921 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb80400ce60 con 0x7fb81c1056a0 2024-03-18T23:11:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb819ffb640 1 -- 172.21.15.132:0/1616457921 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804010450 con 0x7fb81c1056a0 2024-03-18T23:11:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.638+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c1a6dc0 con 0x7fb81c1056a0 2024-03-18T23:11:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.640+0000 7fb819ffb640 1 -- 172.21.15.132:0/1616457921 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fb804005020 con 0x7fb81c1056a0 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb819ffb640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 0x7fb8080a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 Processor -- start 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb819ffb640 1 -- 172.21.15.132:0/1616457921 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8040cf1e0 con 0x7fb81c1056a0 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb820e48640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 0x7fb8080a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 -- start start 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d87d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d8889d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561c4d7e0be0 con 0x561c4d87d810 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c4d789960 con 0x561c4d8865e0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb825d74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c4d7dcc80 con 0x561c4d87cef0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb821649640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d8889d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35578/0 (socket says 172.21.15.132:35578) 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb81bfff640 1 -- 172.21.15.132:0/1745429104 learned_addr learned my addr 172.21.15.132:0/1745429104 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb820e48640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 0x7fb8080a4ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb814006010 tx=0x7fb814005dd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.641+0000 7fb820e48640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d87d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 -- 172.21.15.132:0/1745429104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 msgr2=0x561c4d87d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d87d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 -- 172.21.15.132:0/1745429104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 msgr2=0x561c4d8860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 -- 172.21.15.132:0/1745429104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d780d20 con 0x561c4d8865e0 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb820e48640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d87d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb821649640 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d8889d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cc960 tx=0x7fb80409b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cef61063bd9733f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb812ffd640 1 -- 172.21.15.132:0/1745429104 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80409f9c0 con 0x561c4d8865e0 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb812ffd640 1 -- 172.21.15.132:0/1745429104 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8040a4010 con 0x561c4d8865e0 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.642+0000 7fb812ffd640 1 -- 172.21.15.132:0/1745429104 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804005310 con 0x561c4d8865e0 2024-03-18T23:11:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- 172.21.15.132:0/1745429104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 msgr2=0x561c4d8889d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d8889d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb8040cc960 tx=0x7fb80409b1d0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- 172.21.15.132:0/1745429104 shutdown_connections 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d8889d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d8860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- 172.21.15.132:0/1745429104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d87d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- 172.21.15.132:0/1745429104 >> 172.21.15.132:0/1745429104 conn(0x561c4d877810 msgr2=0x561c4d877bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- 172.21.15.132:0/1745429104 shutdown_connections 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- 172.21.15.132:0/1745429104 wait complete. 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 -- start start 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 Processor -- start 2024-03-18T23:11:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d885fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.643+0000 7fb825d74740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d883340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d883950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561c4d889210 con 0x561c4d87d810 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c4d79cdb0 con 0x561c4d8865e0 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561c4d889390 con 0x561c4d87cef0 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 10 client.0 _ll_register_callbacks cb 0x561c4d7a1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d883340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d883340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35584/0 (socket says 172.21.15.132:35584) 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb81bfff640 1 -- 172.21.15.132:0/3522462403 learned_addr learned my addr 172.21.15.132:0/3522462403 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d883950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb820e48640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d885fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 -- 172.21.15.132:0/3522462403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 msgr2=0x561c4d885fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d885fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 -- 172.21.15.132:0/3522462403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 msgr2=0x561c4d883340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d883340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 -- 172.21.15.132:0/3522462403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4d79bc60 con 0x561c4d8865e0 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb821649640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d883950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb804099b20 tx=0x7fb804086350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aad1a30ea71543d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:11:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804005310 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8040a3b80 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8040a4010 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8040054b0 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 10 client.25361 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.644+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c4d4f5210 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8040a4010 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fb80409ad50 con 0x561c4d8865e0 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 10 client.25361 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 20 client.25361 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d896320 0x561c4d898710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 4 client.25361 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c4d7ee120 con 0x561c4d896320 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb820e48640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d896320 0x561c4d898710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb7fbfff640 10 client.25361 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.645+0000 7fb820e48640 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d896320 0x561c4d898710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39676d2004bda96e server_cookie=3761b4cb01c2dfc9 in_seq=0 out_seq=0 2024-03-18T23:11:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb7fbfff640 1 -- 172.21.15.132:0/3522462403 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c4d7ee120 con 0x561c4d896320 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb7fbfff640 10 client.25361 handle_command_reply: tid=0 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 client.25361 shutdown 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 msgr2=0x561c4d883950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d883950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb804099b20 tx=0x7fb804086350 comp rx=0 tx=0).stop 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 shutdown_connections 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d896320 0x561c4d898710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d8865e0 0x561c4d883950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb7fbfff640 0 client.25361 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87d810 0x561c4d883340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 --2- 172.21.15.132:0/3522462403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87cef0 0x561c4d885fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 >> 172.21.15.132:0/3522462403 conn(0x561c4d877810 msgr2=0x561c4d876ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 shutdown_connections 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.646+0000 7fb825d74740 1 -- 172.21.15.132:0/3522462403 wait complete. 2024-03-18T23:11:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.647+0000 7fb825d74740 20 client.25361 trim_cache size 0 max 16384 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 Processor -- start 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 -- start start 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 0x561c4d8832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 0x561c4d885a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1b1e90 con 0x561c4d87cef0 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1af760 con 0x561c4d87d810 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb825d74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ad5b0 con 0x561c4d877810 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.661+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35594/0 (socket says 172.21.15.132:35594) 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 -- 172.21.15.132:0/1490146069 learned_addr learned my addr 172.21.15.132:0/1490146069 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb820e48640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 0x561c4d8832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb821649640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 0x561c4d885a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 -- 172.21.15.132:0/1490146069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 msgr2=0x561c4d8832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 0x561c4d8832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 -- 172.21.15.132:0/1490146069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 msgr2=0x561c4d885a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 0x561c4d885a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 -- 172.21.15.132:0/1490146069 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d5c1620 con 0x561c4d87cef0 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb820e48640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 0x561c4d8832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb821649640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 0x561c4d885a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb81bfff640 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00fa30 tx=0x7fb80c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a00d663121150e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb8127fc640 1 -- 172.21.15.132:0/1490146069 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c022070 con 0x561c4d87cef0 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb8127fc640 1 -- 172.21.15.132:0/1490146069 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb80c003970 con 0x561c4d87cef0 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.662+0000 7fb8127fc640 1 -- 172.21.15.132:0/1490146069 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004700 con 0x561c4d87cef0 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- 172.21.15.132:0/1490146069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 msgr2=0x561c4d89b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00fa30 tx=0x7fb80c009500 comp rx=0 tx=0).stop 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- 172.21.15.132:0/1490146069 shutdown_connections 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d87d810 0x561c4d885a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d89b730 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 --2- 172.21.15.132:0/1490146069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d877810 0x561c4d8832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- 172.21.15.132:0/1490146069 >> 172.21.15.132:0/1490146069 conn(0x561c4d876e40 msgr2=0x561c4d882bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- 172.21.15.132:0/1490146069 shutdown_connections 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- 172.21.15.132:0/1490146069 wait complete. 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 -- start start 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.663+0000 7fb825d74740 1 Processor -- start 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 0x561c4d899180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87d810 0x561c4d899c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1adc20 con 0x561c4d87cef0 2024-03-18T23:11:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1af090 con 0x561c4d877810 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1ae2f0 con 0x561c4d87d810 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 10 client.0 _ll_register_callbacks cb 0x7fb80c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb820e48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 0x561c4d899180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35608/0 (socket says 172.21.15.132:35608) 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 -- 172.21.15.132:0/157188440 learned_addr learned my addr 172.21.15.132:0/157188440 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb821649640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87d810 0x561c4d899c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 -- 172.21.15.132:0/157188440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87d810 msgr2=0x561c4d899c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87d810 0x561c4d899c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 -- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 msgr2=0x561c4d899180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 0x561c4d899180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 -- 172.21.15.132:0/157188440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4d8fb3f0 con 0x561c4d87cef0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb820e48640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 0x561c4d899180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb81bfff640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000f80 tx=0x7fb80c012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee54134bea566da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c022070 con 0x561c4d87cef0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb80c004c50 con 0x561c4d87cef0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80c003970 con 0x561c4d87cef0 2024-03-18T23:11:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb80c004e50 con 0x561c4d87cef0 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 10 client.15522 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.664+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c4d8fc6e0 con 0x561c4d87cef0 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb80c003970 con 0x561c4d87cef0 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fb80c028070 con 0x561c4d87cef0 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 10 client.15522 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 20 client.15522 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d8865e0 0x561c4d8fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 4 client.15522 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c4d7e8470 con 0x561c4d8865e0 2024-03-18T23:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb820e48640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d8865e0 0x561c4d8fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb7fbfff640 10 client.15522 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.665+0000 7fb820e48640 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d8865e0 0x561c4d8fedd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66c2b79d5876bb3 server_cookie=fef810f7246765b7 in_seq=0 out_seq=0 2024-03-18T23:11:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb7fbfff640 1 -- 172.21.15.132:0/157188440 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c4d7e8470 con 0x561c4d8865e0 2024-03-18T23:11:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb7fbfff640 10 client.15522 handle_command_reply: tid=0 2024-03-18T23:11:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 client.15522 shutdown 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 msgr2=0x561c4d8996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb80c000f80 tx=0x7fb80c012ca0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 shutdown_connections 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561c4d8865e0 0x561c4d8fedd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561c4d87d810 0x561c4d899c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c4d87cef0 0x561c4d8996c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb7fbfff640 0 client.15522 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 --2- 172.21.15.132:0/157188440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c4d877810 0x561c4d899180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.672+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 >> 172.21.15.132:0/157188440 conn(0x561c4d876e40 msgr2=0x561c4d878c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 shutdown_connections 2024-03-18T23:11:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb825d74740 1 -- 172.21.15.132:0/157188440 wait complete. 2024-03-18T23:11:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb825d74740 20 client.15522 trim_cache size 0 max 16384 2024-03-18T23:11:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:11:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:11:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:11:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:11:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 msgr2=0x7fb8080a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 0x7fb8080a4ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb814006010 tx=0x7fb814005dd0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 msgr2=0x7fb81c10eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e970 tx=0x7fb80400ed70 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 shutdown_connections 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8080a2a10 0x7fb8080a4ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb81c1056a0 0x7fb81c10eae0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c101310 0x7fb81c10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 --2- 172.21.15.132:0/1616457921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb81c100970 0x7fb81c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 >> 172.21.15.132:0/1616457921 conn(0x7fb81c0774f0 msgr2=0x7fb81c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 shutdown_connections 2024-03-18T23:11:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:02.673+0000 7fb8230ac640 1 -- 172.21.15.132:0/1616457921 wait complete. 2024-03-18T23:11:03.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:03 smithi144 ceph-mon[66781]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:03.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:03 smithi132 ceph-mon[32020]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:03 smithi196 ceph-mon[67089]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:05.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:05 smithi144 ceph-mon[66781]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:05.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:05 smithi132 ceph-mon[32020]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:05 smithi196 ceph-mon[67089]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:11:07.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:07 smithi144 ceph-mon[66781]: pgmap v59: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:07 smithi132 ceph-mon[32020]: pgmap v59: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:07.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:07 smithi196 ceph-mon[67089]: pgmap v59: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:11:09.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:09 smithi144 ceph-mon[66781]: pgmap v60: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-03-18T23:11:09.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:09 smithi132 ceph-mon[32020]: pgmap v60: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-03-18T23:11:09.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:09 smithi196 ceph-mon[67089]: pgmap v60: 97 pgs: 97 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-03-18T23:11:11.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:11 smithi144 ceph-mon[66781]: pgmap v61: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:11:11.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:11 smithi132 ceph-mon[32020]: pgmap v61: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:11:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:11 smithi196 ceph-mon[67089]: pgmap v61: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:11:13.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:13 smithi144 ceph-mon[66781]: pgmap v62: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:13.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:13 smithi132 ceph-mon[32020]: pgmap v62: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:13 smithi196 ceph-mon[67089]: pgmap v62: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:15.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:15 smithi144 ceph-mon[66781]: pgmap v63: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:15.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:15 smithi132 ceph-mon[32020]: pgmap v63: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:15 smithi196 ceph-mon[67089]: pgmap v63: 97 pgs: 97 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 -- 172.21.15.132:0/3999843367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98074510 msgr2=0x7f0f9810db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 --2- 172.21.15.132:0/3999843367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98074510 0x7f0f9810db20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009900 tx=0x7f0f8c02bcd0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 -- 172.21.15.132:0/3999843367 shutdown_connections 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 --2- 172.21.15.132:0/3999843367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 0x7f0f98110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 --2- 172.21.15.132:0/3999843367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98074510 0x7f0f9810db20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 --2- 172.21.15.132:0/3999843367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98073bf0 0x7f0f98073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 -- 172.21.15.132:0/3999843367 >> 172.21.15.132:0/3999843367 conn(0x7f0f980fd2b0 msgr2=0x7f0f980ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 -- 172.21.15.132:0/3999843367 shutdown_connections 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.769+0000 7f0f9e137640 1 -- 172.21.15.132:0/3999843367 wait complete. 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 Processor -- start 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 -- start start 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98074510 0x7f0f9819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 0x7f0f9819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98113de0 con 0x7f0f98073bf0 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98113c60 con 0x7f0f98074510 2024-03-18T23:11:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f9e137640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98113f60 con 0x7f0f9810e060 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41528/0 (socket says 172.21.15.132:41528) 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f977fe640 1 -- 172.21.15.132:0/1541080874 learned_addr learned my addr 172.21.15.132:0/1541080874 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98074510 0x7f0f9819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.770+0000 7f0f97fff640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 0x7f0f9819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 -- 172.21.15.132:0/1541080874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 msgr2=0x7f0f9819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 0x7f0f9819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 -- 172.21.15.132:0/1541080874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98074510 msgr2=0x7f0f9819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98074510 0x7f0f9819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 -- 172.21.15.132:0/1541080874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9819f280 con 0x7f0f98073bf0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f977fe640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800ea60 tx=0x7f0f8800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f94ff9640 1 -- 172.21.15.132:0/1541080874 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800cde0 con 0x7f0f98073bf0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9819f570 con 0x7f0f98073bf0 2024-03-18T23:11:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f94ff9640 1 -- 172.21.15.132:0/1541080874 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f880035f0 con 0x7f0f98073bf0 2024-03-18T23:11:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f94ff9640 1 -- 172.21.15.132:0/1541080874 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88010670 con 0x7f0f98073bf0 2024-03-18T23:11:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.771+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f981a6e00 con 0x7f0f98073bf0 2024-03-18T23:11:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.773+0000 7f0f94ff9640 1 -- 172.21.15.132:0/1541080874 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f0f880037a0 con 0x7f0f98073bf0 2024-03-18T23:11:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 Processor -- start 2024-03-18T23:11:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 -- start start 2024-03-18T23:11:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c170e5e0 0x5651c17109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5651c1668be0 con 0x5651c170e5e0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651c1611960 con 0x5651c1704ef0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651c1664c80 con 0x5651c1705810 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37628/0 (socket says 172.21.15.132:37628) 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c170e5e0 0x5651c17109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 -- 172.21.15.132:0/2334924596 learned_addr learned my addr 172.21.15.132:0/2334924596 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f94ff9640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 0x7f0f800a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 0x7f0f800a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 -- 172.21.15.132:0/2334924596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 msgr2=0x5651c170e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 -- 172.21.15.132:0/2334924596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c170e5e0 msgr2=0x5651c17109d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c170e5e0 0x5651c17109d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f97fff640 1 -- 172.21.15.132:0/2334924596 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c1608d20 con 0x5651c1704ef0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f977fe640 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.774+0000 7f0f94ff9640 1 -- 172.21.15.132:0/1541080874 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f0f880cf0c0 con 0x7f0f98073bf0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0f97fff640 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f900094d0 tx=0x7f0f90013eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb63cdd49d310083 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0f7dffb640 1 -- 172.21.15.132:0/2334924596 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90022020 con 0x5651c1704ef0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0f97fff640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 0x7f0f800a4ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f90023000 tx=0x7f0f90021000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0f7dffb640 1 -- 172.21.15.132:0/2334924596 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90012d00 con 0x5651c1704ef0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0f7dffb640 1 -- 172.21.15.132:0/2334924596 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90010690 con 0x5651c1704ef0 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2334924596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 msgr2=0x5651c17052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f900094d0 tx=0x7f0f90013eb0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2334924596 shutdown_connections 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c170e5e0 0x5651c17109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.775+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2334924596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1704ef0 0x5651c17052d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2334924596 >> 172.21.15.132:0/2334924596 conn(0x5651c16ff810 msgr2=0x5651c16ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2334924596 shutdown_connections 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2334924596 wait complete. 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- start start 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 Processor -- start 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 0x5651c170b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5651c1711210 con 0x5651c1704ef0 2024-03-18T23:11:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651c1624db0 con 0x5651c170e5e0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.776+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651c1711390 con 0x5651c1705810 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0fa0df8740 10 client.0 _ll_register_callbacks cb 0x5651c1629e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41550/0 (socket says 172.21.15.132:41550) 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 -- 172.21.15.132:0/2400680530 learned_addr learned my addr 172.21.15.132:0/2400680530 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f97fff640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 0x5651c170b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 -- 172.21.15.132:0/2400680530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 msgr2=0x5651c170b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 -- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 msgr2=0x5651c170b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 0x5651c170b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 -- 172.21.15.132:0/2400680530 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c1623c60 con 0x5651c1704ef0 2024-03-18T23:11:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f97fff640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 0x5651c170b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f977fe640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f88097bb0 tx=0x7f0f880d2720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca35ee1c0ca3414 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8809be50 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f0f880ccac0 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f90012d00 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0f8809b750 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0fa0df8740 10 client.15534 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.777+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5651c137d210 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90012d00 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f0f88064190 con 0x5651c1704ef0 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 10 client.15534 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 20 client.15534 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c171e470 0x5651c1720860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 4 client.15534 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651c1676120 con 0x5651c171e470 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c171e470 0x5651c1720860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0f76ffd640 10 client.15534 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.778+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c171e470 0x5651c1720860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5709ac771c491a7 server_cookie=91334e25b9d427d2 in_seq=0 out_seq=0 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0f76ffd640 1 -- 172.21.15.132:0/2400680530 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5651c1676120 con 0x5651c171e470 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0f76ffd640 10 client.15534 handle_command_reply: tid=0 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 client.15534 shutdown 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 msgr2=0x5651c170df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f88097bb0 tx=0x7f0f880d2720 comp rx=0 tx=0).stop 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 shutdown_connections 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c171e470 0x5651c1720860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c170e5e0 0x5651c170b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c1705810 0x5651c170b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0f76ffd640 0 client.15534 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/2400680530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c170df90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 >> 172.21.15.132:0/2400680530 conn(0x5651c16ff810 msgr2=0x5651c16feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 shutdown_connections 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 1 -- 172.21.15.132:0/2400680530 wait complete. 2024-03-18T23:11:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.779+0000 7f0fa0df8740 20 client.15534 trim_cache size 0 max 16384 2024-03-18T23:11:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 Processor -- start 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 -- start start 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1726120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c170d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b1f30 con 0x5651c1704ef0 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981af800 con 0x5651c1705810 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.794+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981ad650 con 0x5651c170e5e0 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1726120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41562/0 (socket says 172.21.15.132:41562) 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f977fe640 1 -- 172.21.15.132:0/758075550 learned_addr learned my addr 172.21.15.132:0/758075550 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f97fff640 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c170d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 -- 172.21.15.132:0/758075550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 msgr2=0x5651c170d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c170d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 -- 172.21.15.132:0/758075550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 msgr2=0x5651c1719fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 -- 172.21.15.132:0/758075550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c1705640 con 0x5651c1705810 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f977fe640 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1726120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000c00 tx=0x7f0f8c02c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d66d73cbff6b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f7d7fa640 1 -- 172.21.15.132:0/758075550 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c050070 con 0x5651c1705810 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f7d7fa640 1 -- 172.21.15.132:0/758075550 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0042c0 con 0x5651c1705810 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.795+0000 7f0f7d7fa640 1 -- 172.21.15.132:0/758075550 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c004f10 con 0x5651c1705810 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- 172.21.15.132:0/758075550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 msgr2=0x5651c1726120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1726120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000c00 tx=0x7f0f8c02c480 comp rx=0 tx=0).stop 2024-03-18T23:11:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- 172.21.15.132:0/758075550 shutdown_connections 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c170d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1726120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/758075550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1719fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- 172.21.15.132:0/758075550 >> 172.21.15.132:0/758075550 conn(0x5651c16ff810 msgr2=0x5651c16feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- 172.21.15.132:0/758075550 shutdown_connections 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- 172.21.15.132:0/758075550 wait complete. 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 -- start start 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.796+0000 7f0fa0df8740 1 Processor -- start 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c1783750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981adcc0 con 0x5651c1704ef0 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981af130 con 0x5651c1705810 2024-03-18T23:11:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981ae390 con 0x5651c170e5e0 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 10 client.0 _ll_register_callbacks cb 0x7f0f8800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37656/0 (socket says 172.21.15.132:37656) 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41568/0 (socket says 172.21.15.132:41568) 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 -- 172.21.15.132:0/918909136 learned_addr learned my addr 172.21.15.132:0/918909136 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c1783750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 -- 172.21.15.132:0/918909136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 msgr2=0x5651c1783750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c1783750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 -- 172.21.15.132:0/918909136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 msgr2=0x5651c1708650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 -- 172.21.15.132:0/918909136 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c1449620 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f977fe640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f96ffd640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009850 tx=0x7f0f8c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c40adf563b79a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c050070 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f0f8c004c60 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.797+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c0042c0 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0f8c02c920 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 10 client.25385 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5651c1784dd0 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0042c0 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f0f8c04a080 con 0x5651c1705810 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 10 client.25385 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 20 client.25385 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c1785460 0x5651c1787880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 4 client.25385 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651c1670470 con 0x5651c1785460 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.798+0000 7f0f977fe640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c1785460 0x5651c1787880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.799+0000 7f0f76ffd640 10 client.25385 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.799+0000 7f0f977fe640 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c1785460 0x5651c1787880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbfc3545a62daa60 server_cookie=3a06a2cbab3a31ca in_seq=0 out_seq=0 2024-03-18T23:11:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0f76ffd640 1 -- 172.21.15.132:0/918909136 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651c1670470 con 0x5651c1785460 2024-03-18T23:11:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0f76ffd640 10 client.25385 handle_command_reply: tid=0 2024-03-18T23:11:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 client.25385 shutdown 2024-03-18T23:11:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 msgr2=0x5651c1708b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009850 tx=0x7f0f8c0040c0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 shutdown_connections 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5651c1785460 0x5651c1787880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5651c170e5e0 0x5651c1783750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651c1705810 0x5651c1708b90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0f76ffd640 0 client.25385 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 --2- 172.21.15.132:0/918909136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5651c1704ef0 0x5651c1708650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 >> 172.21.15.132:0/918909136 conn(0x5651c16ff810 msgr2=0x5651c1700af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.805+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 shutdown_connections 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0fa0df8740 1 -- 172.21.15.132:0/918909136 wait complete. 2024-03-18T23:11:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0fa0df8740 20 client.25385 trim_cache size 0 max 16384 2024-03-18T23:11:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:11:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:11:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:11:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 msgr2=0x7f0f800a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 0x7f0f800a4ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f90023000 tx=0x7f0f90021000 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 msgr2=0x7f0f9819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800ea60 tx=0x7f0f8800ef20 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 shutdown_connections 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0f800a2a10 0x7f0f800a4ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f9810e060 0x7f0f9819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f98074510 0x7f0f9819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 --2- 172.21.15.132:0/1541080874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f98073bf0 0x7f0f9819a2e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 >> 172.21.15.132:0/1541080874 conn(0x7f0f980fd2b0 msgr2=0x7f0f980fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 shutdown_connections 2024-03-18T23:11:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:15.806+0000 7f0f9e137640 1 -- 172.21.15.132:0/1541080874 wait complete. 2024-03-18T23:11:17.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:17 smithi144 ceph-mon[66781]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:17 smithi132 ceph-mon[32020]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:17.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:17 smithi196 ceph-mon[67089]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:19.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:19 smithi144 ceph-mon[66781]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:19.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:19 smithi132 ceph-mon[32020]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:19 smithi196 ceph-mon[67089]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:21.526 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:21 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:21.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:21 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:21 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:11:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:22 smithi144 ceph-mon[66781]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:22 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:22 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:22 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:22 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:22 smithi132 ceph-mon[32020]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:22.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:22 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:22 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:22 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:22 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:22 smithi196 ceph-mon[67089]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:22.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:22 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:22 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:22 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:22.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:22 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:11:23.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:11:23.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:11:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:24 smithi144 ceph-mon[66781]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:24 smithi132 ceph-mon[32020]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:24 smithi196 ceph-mon[67089]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:25 smithi144 ceph-mon[66781]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:25 smithi132 ceph-mon[32020]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:25 smithi196 ceph-mon[67089]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:27.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:27 smithi144 ceph-mon[66781]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:27.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:27 smithi132 ceph-mon[32020]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:27 smithi196 ceph-mon[67089]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.901+0000 7f78986b2640 1 -- 172.21.15.132:0/2874714665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890069840 msgr2=0x7f789010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.901+0000 7f78986b2640 1 --2- 172.21.15.132:0/2874714665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890069840 0x7f789010e670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7888009ca0 tx=0x7f788802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- 172.21.15.132:0/2874714665 shutdown_connections 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- 172.21.15.132:0/2874714665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890069840 0x7f789010e670 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- 172.21.15.132:0/2874714665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890068ea0 0x7f7890069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- 172.21.15.132:0/2874714665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 0x7f7890107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- 172.21.15.132:0/2874714665 >> 172.21.15.132:0/2874714665 conn(0x7f7890077570 msgr2=0x7f7890077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- 172.21.15.132:0/2874714665 shutdown_connections 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- 172.21.15.132:0/2874714665 wait complete. 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 Processor -- start 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- start start 2024-03-18T23:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 0x7f7890100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 0x7f78901014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7890111bb0 con 0x7f7890068ea0 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7890111a30 con 0x7f7890069840 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.902+0000 7f78986b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7890111d30 con 0x7f7890107890 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59334/0 (socket says 172.21.15.132:59334) 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 -- 172.21.15.132:0/713033672 learned_addr learned my addr 172.21.15.132:0/713033672 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f7895c4d640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 0x7f7890100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f7896c4f640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 0x7f78901014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 -- 172.21.15.132:0/713033672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 msgr2=0x7f78901014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 0x7f78901014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 -- 172.21.15.132:0/713033672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 msgr2=0x7f7890100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 0x7f7890100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 -- 172.21.15.132:0/713033672 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901a63d0 con 0x7f7890068ea0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f7895c4d640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 0x7f7890100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f789644e640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f788c00ea60 tx=0x7f788c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f78877fe640 1 -- 172.21.15.132:0/713033672 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00cde0 con 0x7f7890068ea0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f78877fe640 1 -- 172.21.15.132:0/713033672 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c0035f0 con 0x7f7890068ea0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901a66c0 con 0x7f7890068ea0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f78877fe640 1 -- 172.21.15.132:0/713033672 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c010670 con 0x7f7890068ea0 2024-03-18T23:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.903+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901a6c60 con 0x7f7890068ea0 2024-03-18T23:11:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.905+0000 7f78877fe640 1 -- 172.21.15.132:0/713033672 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f788c0037a0 con 0x7f7890068ea0 2024-03-18T23:11:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 Processor -- start 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 -- start start 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620ead9ef0 0x55620eada2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eae30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 0x55620eae59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f78877fe640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 0x7f787c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55620ea3dbe0 con 0x55620eada810 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55620e9e6960 con 0x55620eae35e0 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789b37d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55620ea39c80 con 0x55620ead9ef0 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789644e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eae30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f7896c4f640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 0x7f787c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f7895c4d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 0x55620eae59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789644e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eae30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59350/0 (socket says 172.21.15.132:59350) 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f789644e640 1 -- 172.21.15.132:0/2876924685 learned_addr learned my addr 172.21.15.132:0/2876924685 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.906+0000 7f7896c4f640 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620ead9ef0 0x55620eada2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f78877fe640 1 -- 172.21.15.132:0/713033672 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f788c0cf030 con 0x7f7890068ea0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 -- 172.21.15.132:0/2876924685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620ead9ef0 msgr2=0x55620eada2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620ead9ef0 0x55620eada2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 -- 172.21.15.132:0/2876924685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 msgr2=0x55620eae30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eae30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 -- 172.21.15.132:0/2876924685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620e9ddd20 con 0x55620eae35e0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7896c4f640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 0x7f787c0a4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7888004af0 tx=0x7f7888032340 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7895c4d640 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 0x55620eae59d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7880009fa0 tx=0x7f78800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79020a1185bfa1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7884ff9640 1 -- 172.21.15.132:0/2876924685 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880004020 con 0x55620eae35e0 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7884ff9640 1 -- 172.21.15.132:0/2876924685 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78800041c0 con 0x55620eae35e0 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.907+0000 7f7884ff9640 1 -- 172.21.15.132:0/2876924685 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788000f820 con 0x55620eae35e0 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- 172.21.15.132:0/2876924685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 msgr2=0x55620eae59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 0x55620eae59d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7880009fa0 tx=0x7f78800118d0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- 172.21.15.132:0/2876924685 shutdown_connections 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620eae35e0 0x55620eae59d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eae30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 --2- 172.21.15.132:0/2876924685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620ead9ef0 0x55620eada2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- 172.21.15.132:0/2876924685 >> 172.21.15.132:0/2876924685 conn(0x55620ead4810 msgr2=0x55620ead4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- 172.21.15.132:0/2876924685 shutdown_connections 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- 172.21.15.132:0/2876924685 wait complete. 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 -- start start 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.908+0000 7f789b37d740 1 Processor -- start 2024-03-18T23:11:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eae03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eae0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55620eae6210 con 0x55620eae35e0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55620e9f9db0 con 0x55620ead9ef0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55620eae6390 con 0x55620eada810 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 10 client.0 _ll_register_callbacks cb 0x55620e9fee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7896c4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eae0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7895c4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eae03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:40618/0 (socket says 172.21.15.132:40618) 2024-03-18T23:11:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 -- 172.21.15.132:0/2742590678 learned_addr learned my addr 172.21.15.132:0/2742590678 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 -- 172.21.15.132:0/2742590678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 msgr2=0x55620eae03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eae03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 -- 172.21.15.132:0/2742590678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 msgr2=0x55620eae0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eae0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 -- 172.21.15.132:0/2742590678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620e9f8c60 con 0x55620ead9ef0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7896c4f640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eae0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789644e640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d010 tx=0x7f788c06b190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c934ac361d875b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c09bdc0 con 0x55620ead9ef0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f788c097d30 con 0x55620ead9ef0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.909+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78800041c0 con 0x55620ead9ef0 2024-03-18T23:11:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f788c09b6c0 con 0x55620ead9ef0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 10 client.25397 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55620e752210 con 0x55620ead9ef0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78800041c0 con 0x55620ead9ef0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f788c09ab10 con 0x55620ead9ef0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 10 client.25397 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 20 client.25397 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eaf3560 0x55620eaf5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 4 client.25397 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55620ea4b120 con 0x55620eaf3560 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.910+0000 7f7895c4d640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eaf3560 0x55620eaf5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f7879ffb640 10 client.25397 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f7895c4d640 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eaf3560 0x55620eaf5990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd23d4bb223a91f7 server_cookie=5164f9b4b2f8e5c3 in_seq=0 out_seq=0 2024-03-18T23:11:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f7879ffb640 1 -- 172.21.15.132:0/2742590678 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55620ea4b120 con 0x55620eaf3560 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f7879ffb640 10 client.25397 handle_command_reply: tid=0 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 client.25397 shutdown 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 msgr2=0x55620eae3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d010 tx=0x7f788c06b190 comp rx=0 tx=0).stop 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 shutdown_connections 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eaf3560 0x55620eaf5990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eae0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f7879ffb640 0 client.25397 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eae03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 --2- 172.21.15.132:0/2742590678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eae3010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 >> 172.21.15.132:0/2742590678 conn(0x55620ead4810 msgr2=0x55620ead6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 shutdown_connections 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.911+0000 7f789b37d740 1 -- 172.21.15.132:0/2742590678 wait complete. 2024-03-18T23:11:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.912+0000 7f789b37d740 20 client.25397 trim_cache size 0 max 16384 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 Processor -- start 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 -- start start 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eaeefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eae35e0 0x55620eae2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f78901b1f30 con 0x55620eada810 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78901af800 con 0x55620ead9ef0 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f789b37d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78901ad650 con 0x55620eae35e0 2024-03-18T23:11:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.926+0000 7f7895c4d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f789644e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eaeefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59378/0 (socket says 172.21.15.132:59378) 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 -- 172.21.15.132:0/1261917417 learned_addr learned my addr 172.21.15.132:0/1261917417 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7896c4f640 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eae35e0 0x55620eae2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 -- 172.21.15.132:0/1261917417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eae35e0 msgr2=0x55620eae2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eae35e0 0x55620eae2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 -- 172.21.15.132:0/1261917417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 msgr2=0x55620eaeefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eaeefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 -- 172.21.15.132:0/1261917417 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620eae0e50 con 0x55620eada810 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f789644e640 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eaeefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f7895c4d640 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7880003eb0 tx=0x7f7880003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09488d28cf70d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f787bfff640 1 -- 172.21.15.132:0/1261917417 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788001d040 con 0x55620eada810 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f787bfff640 1 -- 172.21.15.132:0/1261917417 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788001e040 con 0x55620eada810 2024-03-18T23:11:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.927+0000 7f787bfff640 1 -- 172.21.15.132:0/1261917417 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880005420 con 0x55620eada810 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- 172.21.15.132:0/1261917417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 msgr2=0x55620eafb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7880003eb0 tx=0x7f7880003ee0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- 172.21.15.132:0/1261917417 shutdown_connections 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eae35e0 0x55620eae2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eada810 0x55620eafb2a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- 172.21.15.132:0/1261917417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eaeefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- 172.21.15.132:0/1261917417 >> 172.21.15.132:0/1261917417 conn(0x55620ead4810 msgr2=0x55620ead3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- 172.21.15.132:0/1261917417 shutdown_connections 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- 172.21.15.132:0/1261917417 wait complete. 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- start start 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 Processor -- start 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eadd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eadd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f78901adcc0 con 0x55620eae35e0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78901af130 con 0x55620ead9ef0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78901ae390 con 0x55620eada810 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789b37d740 10 client.0 _ll_register_callbacks cb 0x7f788c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f7896c4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f789644e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eadd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f7896c4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59384/0 (socket says 172.21.15.132:59384) 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f7896c4f640 1 -- 172.21.15.132:0/1884061698 learned_addr learned my addr 172.21.15.132:0/1884061698 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.928+0000 7f7895c4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eadd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 -- 172.21.15.132:0/1884061698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 msgr2=0x55620eadd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eadd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 -- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 msgr2=0x55620eadd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eadd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 -- 172.21.15.132:0/1884061698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620e81e620 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7895c4d640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eadd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f789644e640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eadd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7896c4f640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7888037040 tx=0x7f7888005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d80e56e7caa3d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7879ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888053070 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f7888004250 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f788001e040 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f788805f050 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f789b37d740 10 client.15558 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55620eb59d50 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.929+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788001e040 con 0x55620eae35e0 2024-03-18T23:11:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f788803f050 con 0x55620eae35e0 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 10 client.15558 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 20 client.15558 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eb5a500 0x55620eb5c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 4 client.15558 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55620ea45470 con 0x55620eb5a500 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789644e640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eb5a500 0x55620eb5c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f789644e640 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eb5a500 0x55620eb5c920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d998bdf58f5ef927 server_cookie=c554513c2ff48a3e in_seq=0 out_seq=0 2024-03-18T23:11:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.930+0000 7f7879ffb640 10 client.15558 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f7879ffb640 1 -- 172.21.15.132:0/1884061698 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55620ea45470 con 0x55620eb5a500 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f7879ffb640 10 client.15558 handle_command_reply: tid=0 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 client.15558 shutdown 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 msgr2=0x55620eb588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7888037040 tx=0x7f7888005c50 comp rx=0 tx=0).stop 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 shutdown_connections 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55620eb5a500 0x55620eb5c920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55620eae35e0 0x55620eb588d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f7879ffb640 0 client.15558 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55620eada810 0x55620eadd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 --2- 172.21.15.132:0/1884061698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55620ead9ef0 0x55620eadd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 >> 172.21.15.132:0/1884061698 conn(0x55620ead4810 msgr2=0x55620ead5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 shutdown_connections 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 1 -- 172.21.15.132:0/1884061698 wait complete. 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.937+0000 7f789b37d740 20 client.15558 trim_cache size 0 max 16384 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 msgr2=0x7f787c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 0x7f787c0a4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7888004af0 tx=0x7f7888032340 comp rx=0 tx=0).stop 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 msgr2=0x7f7890100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f788c00ea60 tx=0x7f788c00ef20 comp rx=0 tx=0).stop 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 shutdown_connections 2024-03-18T23:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f787c0a2ac0 0x7f787c0a4f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890107890 0x7f78901014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7890069840 0x7f7890100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 --2- 172.21.15.132:0/713033672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7890068ea0 0x7f7890100a40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 >> 172.21.15.132:0/713033672 conn(0x7f7890077570 msgr2=0x7f78900ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 shutdown_connections 2024-03-18T23:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:28.938+0000 7f78986b2640 1 -- 172.21.15.132:0/713033672 wait complete. 2024-03-18T23:11:29.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:29 smithi144 ceph-mon[66781]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:11:29.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:29 smithi132 ceph-mon[32020]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:11:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:29 smithi196 ceph-mon[67089]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:11:31.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:31 smithi132 ceph-mon[32020]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:31 smithi196 ceph-mon[67089]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:31 smithi144 ceph-mon[66781]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:33.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:33 smithi144 ceph-mon[66781]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:33.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:33 smithi132 ceph-mon[32020]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:33 smithi196 ceph-mon[67089]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:35.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:35 smithi132 ceph-mon[32020]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:35 smithi196 ceph-mon[67089]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:35.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:35 smithi144 ceph-mon[66781]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.0 KiB/s wr, 11 op/s 2024-03-18T23:11:37.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:37 smithi132 ceph-mon[32020]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:37 smithi196 ceph-mon[67089]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:37.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:37 smithi144 ceph-mon[66781]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:39.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:39 smithi144 ceph-mon[66781]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:39.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:39 smithi132 ceph-mon[32020]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:39 smithi196 ceph-mon[67089]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:41 smithi132 ceph-mon[32020]: pgmap v76: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:41 smithi196 ceph-mon[67089]: pgmap v76: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:41.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:41 smithi144 ceph-mon[66781]: pgmap v76: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 -- 172.21.15.132:0/1673242034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0101310 msgr2=0x7fb8e010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 --2- 172.21.15.132:0/1673242034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0101310 0x7fb8e010d550 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009ca0 tx=0x7fb8d802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 -- 172.21.15.132:0/1673242034 shutdown_connections 2024-03-18T23:11:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 --2- 172.21.15.132:0/1673242034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0101310 0x7fb8e010d550 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 --2- 172.21.15.132:0/1673242034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e0100970 0x7fb8e0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 --2- 172.21.15.132:0/1673242034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e01056a0 0x7fb8e0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 -- 172.21.15.132:0/1673242034 >> 172.21.15.132:0/1673242034 conn(0x7fb8e00774f0 msgr2=0x7fb8e0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.033+0000 7fb8e5722640 1 -- 172.21.15.132:0/1673242034 shutdown_connections 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 -- 172.21.15.132:0/1673242034 wait complete. 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 Processor -- start 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 -- start start 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e0101310 0x7fb8e01028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 0x7fb8e0102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0110bb0 con 0x7fb8e0100970 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0110a30 con 0x7fb8e01056a0 2024-03-18T23:11:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8e5722640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0110d30 con 0x7fb8e0101310 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47692/0 (socket says 172.21.15.132:47692) 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 0x7fb8e0102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.034+0000 7fb8deffd640 1 -- 172.21.15.132:0/2980650514 learned_addr learned my addr 172.21.15.132:0/2980650514 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e0101310 0x7fb8e01028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 -- 172.21.15.132:0/2980650514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e0101310 msgr2=0x7fb8e01028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e0101310 0x7fb8e01028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 -- 172.21.15.132:0/2980650514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 msgr2=0x7fb8e0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 0x7fb8e0102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 -- 172.21.15.132:0/2980650514 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e01a64e0 con 0x7fb8e0100970 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 0x7fb8e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8deffd640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e9f0 tx=0x7fb8d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8dcff9640 1 -- 172.21.15.132:0/2980650514 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000cd80 con 0x7fb8e0100970 2024-03-18T23:11:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8dcff9640 1 -- 172.21.15.132:0/2980650514 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d0005500 con 0x7fb8e0100970 2024-03-18T23:11:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e01a67d0 con 0x7fb8e0100970 2024-03-18T23:11:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8dcff9640 1 -- 172.21.15.132:0/2980650514 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0010690 con 0x7fb8e0100970 2024-03-18T23:11:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.035+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a6cd0 con 0x7fb8e0100970 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.037+0000 7fb8dcff9640 1 -- 172.21.15.132:0/2980650514 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fb8d00046c0 con 0x7fb8e0100970 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 Processor -- start 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 -- start start 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8dcff9640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 0x7fb8cc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55706b300be0 con 0x55706b3a65e0 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55706b2a9960 con 0x55706b39d810 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.038+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55706b2fcc80 con 0x55706b39cef0 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47708/0 (socket says 172.21.15.132:47708) 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8deffd640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 0x7fb8cc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8df7fe640 1 -- 172.21.15.132:0/2388050132 learned_addr learned my addr 172.21.15.132:0/2388050132 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8dcff9640 1 -- 172.21.15.132:0/2980650514 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8d00cf210 con 0x7fb8e0100970 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 -- 172.21.15.132:0/2388050132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 msgr2=0x55706b39d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 -- 172.21.15.132:0/2388050132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 msgr2=0x55706b3a89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 -- 172.21.15.132:0/2388050132 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706b2a0d20 con 0x55706b39d810 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a60a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009bf0 tx=0x7fb8c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcf62fea99c7f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8deffd640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 0x7fb8cc0a4ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00cca40 tx=0x7fb8d0086040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d57fa640 1 -- 172.21.15.132:0/2388050132 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8004050 con 0x55706b39d810 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d57fa640 1 -- 172.21.15.132:0/2388050132 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8005500 con 0x55706b39d810 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.039+0000 7fb8d57fa640 1 -- 172.21.15.132:0/2388050132 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80035f0 con 0x55706b39d810 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- 172.21.15.132:0/2388050132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 msgr2=0x55706b3a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a60a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009bf0 tx=0x7fb8c8011810 comp rx=0 tx=0).stop 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- 172.21.15.132:0/2388050132 shutdown_connections 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a60a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/2388050132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- 172.21.15.132:0/2388050132 >> 172.21.15.132:0/2388050132 conn(0x55706b397810 msgr2=0x55706b397bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- 172.21.15.132:0/2388050132 shutdown_connections 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- 172.21.15.132:0/2388050132 wait complete. 2024-03-18T23:11:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 -- start start 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.040+0000 7fb8e83ed740 1 Processor -- start 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39cef0 0x55706b3a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55706b3a9210 con 0x55706b3a65e0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55706b2bcdb0 con 0x55706b39cef0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55706b3a9390 con 0x55706b39d810 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8e83ed740 10 client.0 _ll_register_callbacks cb 0x55706b2c1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39cef0 0x55706b3a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:48772/0 (socket says 172.21.15.132:48772) 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 -- 172.21.15.132:0/1877157753 learned_addr learned my addr 172.21.15.132:0/1877157753 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 -- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39cef0 msgr2=0x55706b3a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39cef0 0x55706b3a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 -- 172.21.15.132:0/1877157753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 msgr2=0x55706b3a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 -- 172.21.15.132:0/1877157753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706b2bbc60 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8004a20 tx=0x7fb8c8009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16747b1518a7abc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.041+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8005030 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8c80051d0 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c8005500 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8c8032050 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 10 client.25414 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55706b015210 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8005500 con 0x55706b39d810 2024-03-18T23:11:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fb8c8018070 con 0x55706b39d810 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 10 client.25414 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 10 client.25414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 20 client.25414 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b3b6340 0x55706b3b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 4 client.25414 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706b30e120 con 0x55706b3b6340 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.042+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b3b6340 0x55706b3b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8c67fc640 10 client.25414 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b3b6340 0x55706b3b8730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b724c03cbf010e7 server_cookie=b5792a55f5b30632 in_seq=0 out_seq=0 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8c67fc640 1 -- 172.21.15.132:0/1877157753 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55706b30e120 con 0x55706b3b6340 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8c67fc640 10 client.25414 handle_command_reply: tid=0 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 client.25414 shutdown 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 msgr2=0x55706b3a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8004a20 tx=0x7fb8c8009fd0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 shutdown_connections 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b3b6340 0x55706b3b8730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39d810 0x55706b3a32f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8c67fc640 0 client.25414 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/1877157753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39cef0 0x55706b3a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 >> 172.21.15.132:0/1877157753 conn(0x55706b397810 msgr2=0x55706b396ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 shutdown_connections 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 1 -- 172.21.15.132:0/1877157753 wait complete. 2024-03-18T23:11:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.043+0000 7fb8e83ed740 20 client.25414 trim_cache size 0 max 16384 2024-03-18T23:11:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.058+0000 7fb8e83ed740 1 Processor -- start 2024-03-18T23:11:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.058+0000 7fb8e83ed740 1 -- start start 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.058+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b3b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.058+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.058+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b3a65e0 0x55706b3a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01b1e70 con 0x55706b39d810 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01af740 con 0x55706b3a65e0 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01ad590 con 0x55706b39cef0 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47736/0 (socket says 172.21.15.132:47736) 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 -- 172.21.15.132:0/3220481666 learned_addr learned my addr 172.21.15.132:0/3220481666 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b3a65e0 0x55706b3a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b3b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 -- 172.21.15.132:0/3220481666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 msgr2=0x55706b3b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b3b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 -- 172.21.15.132:0/3220481666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b3a65e0 msgr2=0x55706b3a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b3a65e0 0x55706b3a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 -- 172.21.15.132:0/3220481666 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706b0e1620 con 0x55706b39d810 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8000c00 tx=0x7fb8c8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80386be06f851dd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d4ff9640 1 -- 172.21.15.132:0/3220481666 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8004460 con 0x55706b39d810 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d4ff9640 1 -- 172.21.15.132:0/3220481666 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8029070 con 0x55706b39d810 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.059+0000 7fb8d4ff9640 1 -- 172.21.15.132:0/3220481666 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8004a80 con 0x55706b39d810 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3220481666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 msgr2=0x55706b3bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8000c00 tx=0x7fb8c8005d30 comp rx=0 tx=0).stop 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3220481666 shutdown_connections 2024-03-18T23:11:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b3a65e0 0x55706b3a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b39d810 0x55706b3bdf30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3220481666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b3b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3220481666 >> 172.21.15.132:0/3220481666 conn(0x55706b397810 msgr2=0x55706b397bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3220481666 shutdown_connections 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3220481666 wait complete. 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 -- start start 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 Processor -- start 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.060+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01adc00 con 0x55706b3a65e0 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01af070 con 0x55706b39d810 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01ae2d0 con 0x55706b39cef0 2024-03-18T23:11:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 10 client.0 _ll_register_callbacks cb 0x7fb8c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:47750/0 (socket says 172.21.15.132:47750) 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 -- 172.21.15.132:0/3158778041 learned_addr learned my addr 172.21.15.132:0/3158778041 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 -- 172.21.15.132:0/3158778041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 msgr2=0x55706b39fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 -- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 msgr2=0x55706b3a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 -- 172.21.15.132:0/3158778041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706b41b560 con 0x55706b3a65e0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8df7fe640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8005cc0 tx=0x7fb8c80043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9806313d1d3b1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8039040 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8d8003a30 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c8029070 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8d8004b40 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 10 client.15576 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.061+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55706b41c750 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8029070 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7fb8d803f040 con 0x55706b3a65e0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 10 client.15576 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 20 client.15576 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b41ceb0 0x55706b41f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 4 client.15576 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55706b308470 con 0x55706b41ceb0 2024-03-18T23:11:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b41ceb0 0x55706b41f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8c67fc640 10 client.15576 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.062+0000 7fb8d7fff640 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b41ceb0 0x55706b41f2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c83ac8f7f5faba8 server_cookie=cb9ea6510afae7c4 in_seq=0 out_seq=0 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8c67fc640 1 -- 172.21.15.132:0/3158778041 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55706b308470 con 0x55706b41ceb0 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8c67fc640 10 client.15576 handle_command_reply: tid=0 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 client.15576 shutdown 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 msgr2=0x55706b3a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8005cc0 tx=0x7fb8c80043f0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 shutdown_connections 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55706b41ceb0 0x55706b41f2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55706b3a65e0 0x55706b3a07f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55706b39d810 0x55706b3a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8c67fc640 0 client.15576 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 --2- 172.21.15.132:0/3158778041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55706b39cef0 0x55706b39fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 >> 172.21.15.132:0/3158778041 conn(0x55706b397810 msgr2=0x55706b3989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 shutdown_connections 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 1 -- 172.21.15.132:0/3158778041 wait complete. 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.069+0000 7fb8e83ed740 20 client.15576 trim_cache size 0 max 16384 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:11:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:11:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 msgr2=0x7fb8cc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 0x7fb8cc0a4ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00cca40 tx=0x7fb8d0086040 comp rx=0 tx=0).stop 2024-03-18T23:11:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 msgr2=0x7fb8e0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e9f0 tx=0x7fb8d000edf0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 shutdown_connections 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fb8cc0a2a10 0x7fb8cc0a4ed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8e01056a0 0x7fb8e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e0101310 0x7fb8e01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 --2- 172.21.15.132:0/2980650514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e0100970 0x7fb8e0102390 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 >> 172.21.15.132:0/2980650514 conn(0x7fb8e00774f0 msgr2=0x7fb8e01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 shutdown_connections 2024-03-18T23:11:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:42.070+0000 7fb8e5722640 1 -- 172.21.15.132:0/2980650514 wait complete. 2024-03-18T23:11:43.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:43 smithi132 ceph-mon[32020]: pgmap v77: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:43.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:43 smithi196 ceph-mon[67089]: pgmap v77: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:43 smithi144 ceph-mon[66781]: pgmap v77: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:45.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:45 smithi132 ceph-mon[32020]: pgmap v78: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:45 smithi196 ceph-mon[67089]: pgmap v78: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:45 smithi144 ceph-mon[66781]: pgmap v78: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-18T23:11:47.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:47 smithi132 ceph-mon[32020]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:47 smithi196 ceph-mon[67089]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:47 smithi144 ceph-mon[66781]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:11:49.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:49 smithi132 ceph-mon[32020]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:49 smithi196 ceph-mon[67089]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:49 smithi144 ceph-mon[66781]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-18T23:11:51.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:51 smithi132 ceph-mon[32020]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:51 smithi196 ceph-mon[67089]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:51.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:51 smithi144 ceph-mon[66781]: pgmap v81: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:11:53.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:53 smithi132 ceph-mon[32020]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:53 smithi196 ceph-mon[67089]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:53 smithi144 ceph-mon[66781]: pgmap v82: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 -- 172.21.15.132:0/211543864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b01010c0 msgr2=0x7f45b010d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 --2- 172.21.15.132:0/211543864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b01010c0 0x7f45b010d5d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009cd0 tx=0x7f45a802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 -- 172.21.15.132:0/211543864 shutdown_connections 2024-03-18T23:11:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 --2- 172.21.15.132:0/211543864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b01010c0 0x7f45b010d5d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 --2- 172.21.15.132:0/211543864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0100700 0x7f45b0100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 --2- 172.21.15.132:0/211543864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b0106710 0x7f45b0106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 -- 172.21.15.132:0/211543864 >> 172.21.15.132:0/211543864 conn(0x7f45b0077870 msgr2=0x7f45b0077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 -- 172.21.15.132:0/211543864 shutdown_connections 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.166+0000 7f45b7d45640 1 -- 172.21.15.132:0/211543864 wait complete. 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 Processor -- start 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 -- start start 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b01010c0 0x7f45b01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0106710 0x7f45b019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0110b20 con 0x7f45b0100700 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01109a0 con 0x7f45b01010c0 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b7d45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0110ca0 con 0x7f45b0106710 2024-03-18T23:11:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48406/0 (socket says 172.21.15.132:48406) 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 -- 172.21.15.132:0/3780638454 learned_addr learned my addr 172.21.15.132:0/3780638454 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b52e0640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b01010c0 0x7f45b01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b62e2640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0106710 0x7f45b019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 -- 172.21.15.132:0/3780638454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0106710 msgr2=0x7f45b019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0106710 0x7f45b019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 -- 172.21.15.132:0/3780638454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b01010c0 msgr2=0x7f45b01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b01010c0 0x7f45b01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 -- 172.21.15.132:0/3780638454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b019cfa0 con 0x7f45b0100700 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9f0 tx=0x7f45ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.167+0000 7f45a6ffd640 1 -- 172.21.15.132:0/3780638454 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00cd80 con 0x7f45b0100700 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.168+0000 7f45a6ffd640 1 -- 172.21.15.132:0/3780638454 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac010910 con 0x7f45b0100700 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.168+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b019d290 con 0x7f45b0100700 2024-03-18T23:11:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.168+0000 7f45a6ffd640 1 -- 172.21.15.132:0/3780638454 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac018990 con 0x7f45b0100700 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.168+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01a4b20 con 0x7f45b0100700 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.170+0000 7f45a6ffd640 1 -- 172.21.15.132:0/3780638454 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f45ac010ac0 con 0x7f45b0100700 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.170+0000 7f45baa0d740 1 Processor -- start 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 -- start start 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45a6ffd640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 0x7f459c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5584bdda1be0 con 0x5584bde475e0 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5584bdd4a960 con 0x5584bde3def0 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45baa0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584bdd9dc80 con 0x5584bde3e810 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b62e2640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 0x7f459c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45a6ffd640 1 -- 172.21.15.132:0/3780638454 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f45ac0cf040 con 0x7f45b0100700 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b62e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37140/0 (socket says 172.21.15.132:37140) 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b52e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b62e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48408/0 (socket says 172.21.15.132:48408) 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 -- 172.21.15.132:0/1466240279 learned_addr learned my addr 172.21.15.132:0/1466240279 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 -- 172.21.15.132:0/1466240279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 msgr2=0x5584bde470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 -- 172.21.15.132:0/1466240279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 msgr2=0x5584bde499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b5ae1640 1 -- 172.21.15.132:0/1466240279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bdd41d20 con 0x5584bde3def0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.171+0000 7f45b52e0640 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.172+0000 7f45b62e2640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 0x7f459c0a4f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005cc0 tx=0x7f45a8032340 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.172+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45ac098150 tx=0x7f45ac09b1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad44c960c8f15650 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.172+0000 7f459bfff640 1 -- 172.21.15.132:0/1466240279 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac09b780 con 0x5584bde3def0 2024-03-18T23:11:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.172+0000 7f459bfff640 1 -- 172.21.15.132:0/1466240279 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac09f820 con 0x5584bde3def0 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.172+0000 7f459bfff640 1 -- 172.21.15.132:0/1466240279 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0ccbb0 con 0x5584bde3def0 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- 172.21.15.132:0/1466240279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 msgr2=0x5584bde3e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45ac098150 tx=0x7f45ac09b1a0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- 172.21.15.132:0/1466240279 shutdown_connections 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 --2- 172.21.15.132:0/1466240279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde3e2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- 172.21.15.132:0/1466240279 >> 172.21.15.132:0/1466240279 conn(0x5584bde38810 msgr2=0x5584bde38bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- 172.21.15.132:0/1466240279 shutdown_connections 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- 172.21.15.132:0/1466240279 wait complete. 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 -- start start 2024-03-18T23:11:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.173+0000 7f45baa0d740 1 Processor -- start 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde47010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5584bde4a150 con 0x5584bde475e0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5584bdd5ddb0 con 0x5584bde3def0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584bde4a2d0 con 0x5584bde3e810 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 10 client.0 _ll_register_callbacks cb 0x5584bdd62e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde47010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde47010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37144/0 (socket says 172.21.15.132:37144) 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48424/0 (socket says 172.21.15.132:48424) 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b52e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b5ae1640 1 -- 172.21.15.132:0/252268924 learned_addr learned my addr 172.21.15.132:0/252268924 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 -- 172.21.15.132:0/252268924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 msgr2=0x5584bde443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 -- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 msgr2=0x5584bde47010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde47010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 -- 172.21.15.132:0/252268924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bdd5cc60 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45b62e2640 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45a802c630 tx=0x7f45a802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a843d48b5bc95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45997fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8053060 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f45a8004450 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac09f820 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.174+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f45a8039040 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 10 client.15588 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584bdab6210 con 0x5584bde475e0 2024-03-18T23:11:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac09f820 con 0x5584bde475e0 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f45a803f040 con 0x5584bde475e0 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 10 client.15588 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 20 client.15588 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bde57470 0x5584bde59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 4 client.15588 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584bddaf120 con 0x5584bde57470 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bde57470 0x5584bde59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bde57470 0x5584bde59860 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d883ef32198dac6 server_cookie=f291f3684dc2a0a9 in_seq=0 out_seq=0 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.175+0000 7f45997fa640 10 client.15588 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.176+0000 7f45997fa640 1 -- 172.21.15.132:0/252268924 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5584bddaf120 con 0x5584bde57470 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.176+0000 7f45997fa640 10 client.15588 handle_command_reply: tid=0 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.176+0000 7f45baa0d740 1 client.15588 shutdown 2024-03-18T23:11:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 msgr2=0x5584bde44ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45a802c630 tx=0x7f45a802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 shutdown_connections 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bde57470 0x5584bde59860 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde475e0 0x5584bde44ae0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3e810 0x5584bde443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 --2- 172.21.15.132:0/252268924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3def0 0x5584bde47010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45997fa640 0 client.15588 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 >> 172.21.15.132:0/252268924 conn(0x5584bde38810 msgr2=0x5584bde37ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 shutdown_connections 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 1 -- 172.21.15.132:0/252268924 wait complete. 2024-03-18T23:11:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.177+0000 7f45baa0d740 20 client.15588 trim_cache size 0 max 16384 2024-03-18T23:11:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 Processor -- start 2024-03-18T23:11:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 -- start start 2024-03-18T23:11:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3def0 0x5584bde4f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde475e0 0x5584bde5f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01afd50 con 0x5584bde3e810 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ad620 con 0x5584bde475e0 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45baa0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ab470 con 0x5584bde3def0 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3def0 0x5584bde4f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48440/0 (socket says 172.21.15.132:48440) 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 -- 172.21.15.132:0/2239312485 learned_addr learned my addr 172.21.15.132:0/2239312485 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b62e2640 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde475e0 0x5584bde5f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 -- 172.21.15.132:0/2239312485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3def0 msgr2=0x5584bde4f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3def0 0x5584bde4f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 -- 172.21.15.132:0/2239312485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde475e0 msgr2=0x5584bde5f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde475e0 0x5584bde5f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 -- 172.21.15.132:0/2239312485 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584bdb82620 con 0x5584bde3e810 2024-03-18T23:11:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f45b52e0640 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009ea0 tx=0x7f45a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560ad3cde9850267 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.192+0000 7f459b7fe640 1 -- 172.21.15.132:0/2239312485 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0022070 con 0x5584bde3e810 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f459b7fe640 1 -- 172.21.15.132:0/2239312485 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a00035f0 con 0x5584bde3e810 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f459b7fe640 1 -- 172.21.15.132:0/2239312485 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a001d6c0 con 0x5584bde3e810 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- 172.21.15.132:0/2239312485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 msgr2=0x5584bde52ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009ea0 tx=0x7f45a0005f90 comp rx=0 tx=0).stop 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- 172.21.15.132:0/2239312485 shutdown_connections 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde475e0 0x5584bde5f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3e810 0x5584bde52ef0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 --2- 172.21.15.132:0/2239312485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde3def0 0x5584bde4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- 172.21.15.132:0/2239312485 >> 172.21.15.132:0/2239312485 conn(0x5584bde38810 msgr2=0x5584bde37ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- 172.21.15.132:0/2239312485 shutdown_connections 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- 172.21.15.132:0/2239312485 wait complete. 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.193+0000 7f45baa0d740 1 -- start start 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 Processor -- start 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3e810 0x5584bdeb5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde475e0 0x5584bdeb9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01abae0 con 0x5584bde3def0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01acf50 con 0x5584bde3e810 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ac1b0 con 0x5584bde475e0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45baa0d740 10 client.0 _ll_register_callbacks cb 0x7f45a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48442/0 (socket says 172.21.15.132:48442) 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 -- 172.21.15.132:0/312052998 learned_addr learned my addr 172.21.15.132:0/312052998 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b52e0640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3e810 0x5584bdeb5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b62e2640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde475e0 0x5584bdeb9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 -- 172.21.15.132:0/312052998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde475e0 msgr2=0x5584bdeb9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde475e0 0x5584bdeb9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 -- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3e810 msgr2=0x5584bdeb5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3e810 0x5584bdeb5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 -- 172.21.15.132:0/312052998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584bdeb9fa0 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45b5ae1640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f45ac09b1e0 tx=0x7f45ac00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e380389260b675 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.194+0000 7f45997fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0af060 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f45ac099a80 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a00035f0 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f45ac09f820 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 10 client.15594 fetch_fsmap learned FSMap version 22 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584bdec2560 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a00035f0 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f45ac097040 con 0x5584bde3def0 2024-03-18T23:11:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 10 client.15594 fetch_fsmap finished waiting for FSMap version 22 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 20 client.15594 populate_metadata read hostname 'smithi132' 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bdec28a0 0x5584bdec4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 4 client.15594 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584bdda9470 con 0x5584bdec28a0 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.195+0000 7f45b52e0640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bdec28a0 0x5584bdec4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.196+0000 7f45997fa640 10 client.15594 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.196+0000 7f45b52e0640 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bdec28a0 0x5584bdec4cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8b9460d94a5a2c server_cookie=e830cb0416017b76 in_seq=0 out_seq=0 2024-03-18T23:11:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.202+0000 7f45997fa640 1 -- 172.21.15.132:0/312052998 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584bdda9470 con 0x5584bdec28a0 2024-03-18T23:11:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.202+0000 7f45997fa640 10 client.15594 handle_command_reply: tid=0 2024-03-18T23:11:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.202+0000 7f45baa0d740 1 client.15594 shutdown 2024-03-18T23:11:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.202+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 msgr2=0x5584bdeb5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f45ac09b1e0 tx=0x7f45ac00cf20 comp rx=0 tx=0).stop 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 shutdown_connections 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584bdec28a0 0x5584bdec4cc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584bde475e0 0x5584bdeb9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5584bde3e810 0x5584bdeb5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 --2- 172.21.15.132:0/312052998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584bde3def0 0x5584bdeb5950 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45997fa640 0 client.15594 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 >> 172.21.15.132:0/312052998 conn(0x5584bde38810 msgr2=0x5584bde3ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 shutdown_connections 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 1 -- 172.21.15.132:0/312052998 wait complete. 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45baa0d740 20 client.15594 trim_cache size 0 max 16384 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:11:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:11:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:11:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 msgr2=0x7f459c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 0x7f459c0a4f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005cc0 tx=0x7f45a8032340 comp rx=0 tx=0).stop 2024-03-18T23:11:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 msgr2=0x7f45b0197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9f0 tx=0x7f45ac00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 shutdown_connections 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f459c0a2a90 0x7f459c0a4f50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45b0106710 0x7f45b019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b01010c0 0x7f45b01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 --2- 172.21.15.132:0/3780638454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0100700 0x7f45b0197fb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 >> 172.21.15.132:0/3780638454 conn(0x7f45b0077870 msgr2=0x7f45b006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.203+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 shutdown_connections 2024-03-18T23:11:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:11:55.204+0000 7f45b7d45640 1 -- 172.21.15.132:0/3780638454 wait complete. 2024-03-18T23:11:55.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:55 smithi132 ceph-mon[32020]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:55 smithi196 ceph-mon[67089]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:55 smithi144 ceph-mon[66781]: pgmap v83: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:57.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:57 smithi132 ceph-mon[32020]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:57.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:57 smithi144 ceph-mon[66781]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:57 smithi196 ceph-mon[67089]: pgmap v84: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-03-18T23:11:59.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:11:59 smithi132 ceph-mon[32020]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:11:59 smithi196 ceph-mon[67089]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:11:59.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:11:59 smithi144 ceph-mon[66781]: pgmap v85: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:12:01.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:01 smithi132 ceph-mon[32020]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:01.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:01 smithi196 ceph-mon[67089]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:01.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:01 smithi144 ceph-mon[66781]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:64/600: open fd = 3 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:65/600: open fd = 3 2024-03-18T23:12:03.158 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:66/600: open fd = 3 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:67/600: open fd = 3 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:68/600: open fd = 3 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.159 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:69/600: open fd = 3 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:70/600: open fd = 3 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:71/600: open fd = 3 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:72/600: open fd = 3 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.160 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:73/600: open fd = 3 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:74/600: open fd = 3 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:75/600: open fd = 3 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.161 INFO:tasks.workunit.client.0.smithi132.stdout:76/600: open fd = 3 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:77/600: open fd = 3 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:78/600: open fd = 3 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:79/600: open fd = 3 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.162 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:80/600: open fd = 3 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:81/600: open fd = 3 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:82/600: open fd = 3 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.163 INFO:tasks.workunit.client.0.smithi132.stdout:83/600: open fd = 3 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:84/600: open fd = 3 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:85/600: open fd = 3 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:86/600: open fd = 3 2024-03-18T23:12:03.164 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:87/600: open fd = 3 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:88/600: open fd = 3 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:89/600: open fd = 3 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.165 INFO:tasks.workunit.client.0.smithi132.stdout:90/600: open fd = 3 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:91/600: open fd = 3 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:92/600: open fd = 3 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:93/600: open fd = 3 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.166 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:94/600: open fd = 3 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:95/600: open fd = 3 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:96/600: open fd = 3 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.167 INFO:tasks.workunit.client.0.smithi132.stdout:97/600: open fd = 3 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:98/600: open fd = 3 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:99/600: open fd = 3 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:100/600: open fd = 3 2024-03-18T23:12:03.168 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:101/600: open fd = 3 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:102/600: open fd = 3 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:103/600: open fd = 3 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.169 INFO:tasks.workunit.client.0.smithi132.stdout:104/600: open fd = 3 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:105/600: open fd = 3 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:106/600: open fd = 3 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:107/600: open fd = 3 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.170 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:108/600: open fd = 3 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:109/600: open fd = 3 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:110/600: open fd = 3 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.171 INFO:tasks.workunit.client.0.smithi132.stdout:111/600: open fd = 3 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:112/600: open fd = 3 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:113/600: open fd = 3 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:114/600: open fd = 3 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.172 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:115/600: open fd = 3 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:116/600: open fd = 3 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:117/600: open fd = 3 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:118/600: open fd = 3 2024-03-18T23:12:03.173 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:119/600: open fd = 3 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:120/600: open fd = 3 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:121/600: open fd = 3 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.174 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:122/600: open fd = 3 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:123/600: open fd = 3 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:124/600: open fd = 3 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:125/600: open fd = 3 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.175 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:12:03.176 INFO:tasks.workunit.client.0.smithi132.stdout:126/600: open fd = 3 2024-03-18T23:12:03.176 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:12:03.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:03 smithi132 ceph-mon[32020]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:03 smithi196 ceph-mon[67089]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:03.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:03 smithi144 ceph-mon[66781]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:05.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:05 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:05.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:05 smithi196 ceph-mon[67089]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:05 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:05 smithi144 ceph-mon[66781]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:05 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:05 smithi132 ceph-mon[32020]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:12:07.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:07 smithi132 ceph-mon[32020]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:12:07.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:07 smithi196 ceph-mon[67089]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:12:07.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:07 smithi144 ceph-mon[66781]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-03-18T23:12:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 -- 172.21.15.132:0/718956851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 msgr2=0x7fc250105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 --2- 172.21.15.132:0/718956851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc250105a80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc240009900 tx=0x7fc24002bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 -- 172.21.15.132:0/718956851 shutdown_connections 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 --2- 172.21.15.132:0/718956851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250101310 0x7fc25010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 --2- 172.21.15.132:0/718956851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250100970 0x7fc250100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 --2- 172.21.15.132:0/718956851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc250105a80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 -- 172.21.15.132:0/718956851 >> 172.21.15.132:0/718956851 conn(0x7fc2500774f0 msgr2=0x7fc250077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.299+0000 7fc2549f4640 1 -- 172.21.15.132:0/718956851 shutdown_connections 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.300+0000 7fc2549f4640 1 -- 172.21.15.132:0/718956851 wait complete. 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.300+0000 7fc2549f4640 1 Processor -- start 2024-03-18T23:12:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.300+0000 7fc2549f4640 1 -- start start 2024-03-18T23:12:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc25019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc250110bb0 con 0x7fc2501056a0 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc250110a30 con 0x7fc250101310 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc250110d30 con 0x7fc250100970 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc25019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:39958/0 (socket says 172.21.15.132:39958) 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:52646/0 (socket says 172.21.15.132:52646) 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24dd9b640 1 -- 172.21.15.132:0/2670897575 learned_addr learned my addr 172.21.15.132:0/2670897575 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 -- 172.21.15.132:0/2670897575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 msgr2=0x7fc2501974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 -- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 msgr2=0x7fc25019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc25019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 -- 172.21.15.132:0/2670897575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25019bf60 con 0x7fc250100970 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc25019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc24e59c640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc240036040 tx=0x7fc24002cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc23f7fe640 1 -- 172.21.15.132:0/2670897575 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240041070 con 0x7fc250100970 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25019c1f0 con 0x7fc250100970 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc23f7fe640 1 -- 172.21.15.132:0/2670897575 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240004450 con 0x7fc250100970 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.301+0000 7fc23f7fe640 1 -- 172.21.15.132:0/2670897575 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240003d90 con 0x7fc250100970 2024-03-18T23:12:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.302+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501a3ae0 con 0x7fc250100970 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc23f7fe640 1 -- 172.21.15.132:0/2670897575 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fc240044070 con 0x7fc250100970 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 Processor -- start 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 -- start start 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc23f7fe640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 0x7fc2380a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x558469721df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x558469685be0 con 0x55846972d7f0 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55846962e960 con 0x558469721a10 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.304+0000 7fc2575e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558469681c80 con 0x558469722330 2024-03-18T23:12:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x558469721df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 0x7fc2380a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:52658/0 (socket says 172.21.15.132:52658) 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 -- 172.21.15.132:0/1634301110 learned_addr learned my addr 172.21.15.132:0/1634301110 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc23f7fe640 1 -- 172.21.15.132:0/2670897575 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc2400f0730 con 0x7fc250100970 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 -- 172.21.15.132:0/1634301110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 msgr2=0x558469721df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x558469721df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 -- 172.21.15.132:0/1634301110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 msgr2=0x55846972fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 -- 172.21.15.132:0/1634301110 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558469625d20 con 0x558469722330 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x558469721df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 0x7fc2380a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55846971dc30 tx=0x7fc248011440 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2440115f0 tx=0x7fc244011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62edf4cb5d0e5166 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc23cff9640 1 -- 172.21.15.132:0/1634301110 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24401e070 con 0x558469722330 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc23cff9640 1 -- 172.21.15.132:0/1634301110 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24400f660 con 0x558469722330 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.305+0000 7fc23cff9640 1 -- 172.21.15.132:0/1634301110 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244003870 con 0x558469722330 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- 172.21.15.132:0/1634301110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 msgr2=0x55846972d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2440115f0 tx=0x7fc244011ab0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- 172.21.15.132:0/1634301110 shutdown_connections 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972d2b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1634301110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x558469721df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- 172.21.15.132:0/1634301110 >> 172.21.15.132:0/1634301110 conn(0x55846971c810 msgr2=0x55846971cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- 172.21.15.132:0/1634301110 shutdown_connections 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- 172.21.15.132:0/1634301110 wait complete. 2024-03-18T23:12:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.306+0000 7fc2575e4740 1 -- start start 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 Processor -- start 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x55846972d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5584697303b0 con 0x55846972d7f0 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558469641db0 con 0x558469721a10 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558469730530 con 0x558469722330 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc2575e4740 10 client.0 _ll_register_callbacks cb 0x558469646e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53424/0 (socket says 172.21.15.132:53424) 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 -- 172.21.15.132:0/1017030852 learned_addr learned my addr 172.21.15.132:0/1017030852 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x55846972d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 -- 172.21.15.132:0/1017030852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 msgr2=0x55846972a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 -- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 msgr2=0x55846972d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x55846972d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 -- 172.21.15.132:0/1017030852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558469640c60 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x55846972d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc24801f930 tx=0x7fc248004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb8740b10b48179 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.307+0000 7fc235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248026070 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc24801fd30 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24400f660 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc24800e930 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 10 client.15606 fetch_fsmap learned FSMap version 23 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55846939a210 con 0x55846972d7f0 2024-03-18T23:12:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24400f660 con 0x55846972d7f0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 23) ==== 8342+0+0 (secure 0 0 0) 0x7fc24801d040 con 0x55846972d7f0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 10 client.15606 fetch_fsmap finished waiting for FSMap version 23 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 20 client.15606 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55846973d6d0 0x55846973fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 4 client.15606 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558469693120 con 0x55846973d6d0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.308+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55846973d6d0 0x55846973fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55846973d6d0 0x55846973fac0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbc703899e80adf server_cookie=7d02c49f8d6c7eba in_seq=0 out_seq=0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc235ffb640 10 client.15606 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc235ffb640 1 -- 172.21.15.132:0/1017030852 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558469693120 con 0x55846973d6d0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc235ffb640 10 client.15606 handle_command_reply: tid=0 2024-03-18T23:12:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc2575e4740 1 client.15606 shutdown 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 msgr2=0x55846972aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.309+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc24801f930 tx=0x7fc248004ab0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 shutdown_connections 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55846973d6d0 0x55846973fac0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55846972d7f0 0x55846972aa30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x55846972a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc235ffb640 0 client.15606 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1017030852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558469721a10 0x55846972d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 >> 172.21.15.132:0/1017030852 conn(0x55846971c810 msgr2=0x55846971e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 shutdown_connections 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 1 -- 172.21.15.132:0/1017030852 wait complete. 2024-03-18T23:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.310+0000 7fc2575e4740 20 client.15606 trim_cache size 0 max 16384 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.324+0000 7fc2575e4740 1 Processor -- start 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.324+0000 7fc2575e4740 1 -- start start 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x55846971e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501ada40 con 0x558469721a10 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501ab310 con 0x55846972d7f0 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc2575e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a9160 con 0x558469722330 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53428/0 (socket says 172.21.15.132:53428) 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 -- 172.21.15.132:0/136552532 learned_addr learned my addr 172.21.15.132:0/136552532 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x55846971e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 -- 172.21.15.132:0/136552532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 msgr2=0x5584697391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 -- 172.21.15.132:0/136552532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 msgr2=0x55846971e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x55846971e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 -- 172.21.15.132:0/136552532 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846973fe30 con 0x558469721a10 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc24e59c640 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc24002c170 tx=0x7fc2400d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2786b8355d674d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc237fff640 1 -- 172.21.15.132:0/136552532 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24004b040 con 0x558469721a10 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc237fff640 1 -- 172.21.15.132:0/136552532 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2400d3e50 con 0x558469721a10 2024-03-18T23:12:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.325+0000 7fc237fff640 1 -- 172.21.15.132:0/136552532 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400c5670 con 0x558469721a10 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- 172.21.15.132:0/136552532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 msgr2=0x558469735bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc24002c170 tx=0x7fc2400d1040 comp rx=0 tx=0).stop 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- 172.21.15.132:0/136552532 shutdown_connections 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x55846971e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- 172.21.15.132:0/136552532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x558469735bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- 172.21.15.132:0/136552532 >> 172.21.15.132:0/136552532 conn(0x55846971c810 msgr2=0x55846971dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- 172.21.15.132:0/136552532 shutdown_connections 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- 172.21.15.132:0/136552532 wait complete. 2024-03-18T23:12:08.331 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- start start 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 Processor -- start 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x558469723e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.326+0000 7fc2575e4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a97d0 con 0x558469721a10 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501aac40 con 0x55846972d7f0 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501a9ea0 con 0x558469722330 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 10 client.0 _ll_register_callbacks cb 0x7fc248005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53442/0 (socket says 172.21.15.132:53442) 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 -- 172.21.15.132:0/1949065752 learned_addr learned my addr 172.21.15.132:0/1949065752 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24ed9d640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x558469723e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 -- 172.21.15.132:0/1949065752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 msgr2=0x5584697238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697238f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 -- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 msgr2=0x558469723e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x558469723e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 -- 172.21.15.132:0/1949065752 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558469466620 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc24e59c640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2400bcf10 tx=0x7fc2400c1020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96a93c9c27233d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24004b050 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc2400b95f0 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2400d3e50 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2400bc8e0 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 10 client.15612 fetch_fsmap learned FSMap version 23 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.327+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5584697a5c60 con 0x558469721a10 2024-03-18T23:12:08.333 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2400d3e50 con 0x558469721a10 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 23) ==== 8342+0+0 (secure 0 0 0) 0x7fc2400cf2c0 con 0x558469721a10 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 10 client.15612 fetch_fsmap finished waiting for FSMap version 23 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 20 client.15612 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584697a62a0 0x5584697a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 4 client.15612 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55846968d470 con 0x5584697a62a0 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584697a62a0 0x5584697a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc235ffb640 10 client.15612 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.328+0000 7fc24dd9b640 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584697a62a0 0x5584697a86c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69b7c42b98bb167 server_cookie=f214d1f6a1799f34 in_seq=0 out_seq=0 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc235ffb640 1 -- 172.21.15.132:0/1949065752 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55846968d470 con 0x5584697a62a0 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc235ffb640 10 client.15612 handle_command_reply: tid=0 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 client.15612 shutdown 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 msgr2=0x5584697233b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2400bcf10 tx=0x7fc2400c1020 comp rx=0 tx=0).stop 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 shutdown_connections 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5584697a62a0 0x5584697a86c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55846972d7f0 0x558469723e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558469722330 0x5584697238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc235ffb640 0 client.15612 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 --2- 172.21.15.132:0/1949065752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558469721a10 0x5584697233b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 >> 172.21.15.132:0/1949065752 conn(0x55846971c810 msgr2=0x55846971bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 shutdown_connections 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 1 -- 172.21.15.132:0/1949065752 wait complete. 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.335+0000 7fc2575e4740 20 client.15612 trim_cache size 0 max 16384 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:12:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:12:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 msgr2=0x7fc2380a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 0x7fc2380a4ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55846971dc30 tx=0x7fc248011440 comp rx=0 tx=0).stop 2024-03-18T23:12:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 msgr2=0x7fc250196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc240036040 tx=0x7fc24002cb20 comp rx=0 tx=0).stop 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 shutdown_connections 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fc2380a2a10 0x7fc2380a4ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2501056a0 0x7fc25019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc250101310 0x7fc2501974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 --2- 172.21.15.132:0/2670897575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc250100970 0x7fc250196f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 >> 172.21.15.132:0/2670897575 conn(0x7fc2500774f0 msgr2=0x7fc2500fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 shutdown_connections 2024-03-18T23:12:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:08.336+0000 7fc2549f4640 1 -- 172.21.15.132:0/2670897575 wait complete. 2024-03-18T23:12:10.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:10 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:10.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:10 smithi196 ceph-mon[67089]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:12:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:10 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:10.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:10 smithi144 ceph-mon[66781]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:12:10.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:10 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:12:10.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:10 smithi132 ceph-mon[32020]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:12:11.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:11 smithi196 ceph-mon[67089]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:11 smithi144 ceph-mon[66781]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:11.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:11 smithi132 ceph-mon[32020]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:12:13.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:13 smithi132 ceph-mon[32020]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:13.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:13 smithi196 ceph-mon[67089]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:13 smithi144 ceph-mon[66781]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:15 smithi196 ceph-mon[67089]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:15.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:15 smithi144 ceph-mon[66781]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:15.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:15 smithi132 ceph-mon[32020]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:17.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:17 smithi132 ceph-mon[32020]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:17 smithi196 ceph-mon[67089]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:17.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:17 smithi144 ceph-mon[66781]: pgmap v94: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:19.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:19 smithi132 ceph-mon[32020]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:12:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:19 smithi196 ceph-mon[67089]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:12:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:19 smithi144 ceph-mon[66781]: pgmap v95: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:12:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 -- 172.21.15.132:0/3252489895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce580744f0 msgr2=0x7fce5810db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/3252489895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce580744f0 0x7fce5810db50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fce40009900 tx=0x7fce4002bcd0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 -- 172.21.15.132:0/3252489895 shutdown_connections 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/3252489895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce5810e090 0x7fce58110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/3252489895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce580744f0 0x7fce5810db50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/3252489895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 0x7fce58073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 -- 172.21.15.132:0/3252489895 >> 172.21.15.132:0/3252489895 conn(0x7fce580fd2e0 msgr2=0x7fce580ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 -- 172.21.15.132:0/3252489895 shutdown_connections 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.432+0000 7fce5e8f8640 1 -- 172.21.15.132:0/3252489895 wait complete. 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 Processor -- start 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 -- start start 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 0x7fce5819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 0x7fce5819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113db0 con 0x7fce5810e090 2024-03-18T23:12:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113c30 con 0x7fce580744f0 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.433+0000 7fce5e8f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce58113f30 con 0x7fce58073bd0 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce577fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 0x7fce5819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54696/0 (socket says 172.21.15.132:54696) 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 -- 172.21.15.132:0/2865341268 learned_addr learned my addr 172.21.15.132:0/2865341268 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce57fff640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 0x7fce5819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 -- 172.21.15.132:0/2865341268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 msgr2=0x7fce5819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 0x7fce5819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 -- 172.21.15.132:0/2865341268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 msgr2=0x7fce5819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 0x7fce5819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 -- 172.21.15.132:0/2865341268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5819f250 con 0x7fce5810e090 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce577fe640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 0x7fce5819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5ce95640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce5000eb10 tx=0x7fce5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce557fa640 1 -- 172.21.15.132:0/2865341268 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50004b20 con 0x7fce5810e090 2024-03-18T23:12:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce557fa640 1 -- 172.21.15.132:0/2865341268 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50004cc0 con 0x7fce5810e090 2024-03-18T23:12:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce557fa640 1 -- 172.21.15.132:0/2865341268 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50010760 con 0x7fce5810e090 2024-03-18T23:12:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5819f540 con 0x7fce5810e090 2024-03-18T23:12:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.434+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581a6dd0 con 0x7fce5810e090 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce557fa640 1 -- 172.21.15.132:0/2865341268 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fce50005020 con 0x7fce5810e090 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 Processor -- start 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 -- start start 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563b46434be0 con 0x563b464d0ef0 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b463dd960 con 0x563b464d1810 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce615bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563b46430c80 con 0x563b464da5e0 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce557fa640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 0x7fce440a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce577fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:43682/0 (socket says 172.21.15.132:43682) 2024-03-18T23:12:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce57fff640 1 -- 172.21.15.132:0/529961226 learned_addr learned my addr 172.21.15.132:0/529961226 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.437+0000 7fce5ce95640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 0x7fce440a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce577fe640 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce557fa640 1 -- 172.21.15.132:0/2865341268 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fce500cf340 con 0x7fce5810e090 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 -- 172.21.15.132:0/529961226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 msgr2=0x563b464dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 -- 172.21.15.132:0/529961226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 msgr2=0x563b464d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 -- 172.21.15.132:0/529961226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b463d4d20 con 0x563b464d1810 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce57fff640 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce48009d20 tx=0x7fce48011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b550c14ebc49ac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce5ce95640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 0x7fce440a4ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce5009b960 tx=0x7fce5006d040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce4e7fc640 1 -- 172.21.15.132:0/529961226 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4801e070 con 0x563b464d1810 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce4e7fc640 1 -- 172.21.15.132:0/529961226 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48019b10 con 0x563b464d1810 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.438+0000 7fce4e7fc640 1 -- 172.21.15.132:0/529961226 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800eb80 con 0x563b464d1810 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- 172.21.15.132:0/529961226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 msgr2=0x563b464da0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce48009d20 tx=0x7fce48011810 comp rx=0 tx=0).stop 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- 172.21.15.132:0/529961226 shutdown_connections 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464da0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 --2- 172.21.15.132:0/529961226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- 172.21.15.132:0/529961226 >> 172.21.15.132:0/529961226 conn(0x563b464cb810 msgr2=0x563b464cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- 172.21.15.132:0/529961226 shutdown_connections 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- 172.21.15.132:0/529961226 wait complete. 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 -- start start 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.439+0000 7fce615bf740 1 Processor -- start 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563b464dd210 con 0x563b464d0ef0 2024-03-18T23:12:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b463f0db0 con 0x563b464d1810 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563b464dd390 con 0x563b464da5e0 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce615bf740 10 client.0 _ll_register_callbacks cb 0x563b463f5e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce577fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce577fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54722/0 (socket says 172.21.15.132:54722) 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce577fe640 1 -- 172.21.15.132:0/214180643 learned_addr learned my addr 172.21.15.132:0/214180643 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:43688/0 (socket says 172.21.15.132:43688) 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce577fe640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 -- 172.21.15.132:0/214180643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 msgr2=0x563b464d7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 -- 172.21.15.132:0/214180643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 msgr2=0x563b464d9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 -- 172.21.15.132:0/214180643 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b463efc60 con 0x563b464d1810 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce577fe640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce57fff640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce4801c040 tx=0x7fce48019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22970f954ca466d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.440+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4801e070 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce48019b10 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fce4800e880 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce480098f0 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 10 client.25469 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b46149210 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48019b10 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fce480046c0 con 0x563b464d1810 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 10 client.25469 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 20 client.25469 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b464ea3a0 0x563b464ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 4 client.25469 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b46442120 con 0x563b464ea3a0 2024-03-18T23:12:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.441+0000 7fce577fe640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b464ea3a0 0x563b464ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.442+0000 7fce377fe640 10 client.25469 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.442+0000 7fce577fe640 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b464ea3a0 0x563b464ec790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19ca258d65a69f82 server_cookie=2ca4cc4d3c302d0 in_seq=0 out_seq=0 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce377fe640 1 -- 172.21.15.132:0/214180643 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b46442120 con 0x563b464ea3a0 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce377fe640 10 client.25469 handle_command_reply: tid=0 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 client.25469 shutdown 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 msgr2=0x563b464d7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce4801c040 tx=0x7fce48019660 comp rx=0 tx=0).stop 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 shutdown_connections 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b464ea3a0 0x563b464ec790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464d7350 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 --2- 172.21.15.132:0/214180643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce377fe640 0 client.25469 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 >> 172.21.15.132:0/214180643 conn(0x563b464cb810 msgr2=0x563b464caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 shutdown_connections 2024-03-18T23:12:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 1 -- 172.21.15.132:0/214180643 wait complete. 2024-03-18T23:12:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.443+0000 7fce615bf740 20 client.25469 trim_cache size 0 max 16384 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 Processor -- start 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 -- start start 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b1f10 con 0x563b464d0ef0 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce581af7e0 con 0x563b464d1810 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce615bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce581ad630 con 0x563b464da5e0 2024-03-18T23:12:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54736/0 (socket says 172.21.15.132:54736) 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 -- 172.21.15.132:0/2112968814 learned_addr learned my addr 172.21.15.132:0/2112968814 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce57fff640 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 -- 172.21.15.132:0/2112968814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 msgr2=0x563b464d9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 -- 172.21.15.132:0/2112968814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 msgr2=0x563b464f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 -- 172.21.15.132:0/2112968814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b464d7cd0 con 0x563b464d0ef0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce577fe640 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce40036040 tx=0x7fce4002cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e1dcb2053aa5b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.458+0000 7fce4dffb640 1 -- 172.21.15.132:0/2112968814 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400041e0 con 0x563b464d0ef0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce4dffb640 1 -- 172.21.15.132:0/2112968814 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce400517b0 con 0x563b464d0ef0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce4dffb640 1 -- 172.21.15.132:0/2112968814 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40005190 con 0x563b464d0ef0 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- 172.21.15.132:0/2112968814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 msgr2=0x563b464e5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce40036040 tx=0x7fce4002cf90 comp rx=0 tx=0).stop 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- 172.21.15.132:0/2112968814 shutdown_connections 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464da5e0 0x563b464d9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464d1810 0x563b464f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 --2- 172.21.15.132:0/2112968814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d0ef0 0x563b464e5fb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- 172.21.15.132:0/2112968814 >> 172.21.15.132:0/2112968814 conn(0x563b464cb810 msgr2=0x563b464caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- 172.21.15.132:0/2112968814 shutdown_connections 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- 172.21.15.132:0/2112968814 wait complete. 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.459+0000 7fce615bf740 1 -- start start 2024-03-18T23:12:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 Processor -- start 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464d0ef0 0x563b464d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464da5e0 0x563b464d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fce581adca0 con 0x563b464d1810 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce581af110 con 0x563b464da5e0 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce581ae370 con 0x563b464d0ef0 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce615bf740 10 client.0 _ll_register_callbacks cb 0x7fce4801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54746/0 (socket says 172.21.15.132:54746) 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 -- 172.21.15.132:0/480416291 learned_addr learned my addr 172.21.15.132:0/480416291 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce577fe640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464da5e0 0x563b464d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce577fe640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464d0ef0 0x563b464d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 -- 172.21.15.132:0/480416291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464d0ef0 msgr2=0x563b464d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464d0ef0 0x563b464d4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 -- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464da5e0 msgr2=0x563b464d4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464da5e0 0x563b464d4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 -- 172.21.15.132:0/480416291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b46215620 con 0x563b464d1810 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce57fff640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce480059f0 tx=0x7fce48009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951a403608749618 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.460+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4803b060 con 0x563b464d1810 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fce48019bf0 con 0x563b464d1810 2024-03-18T23:12:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce400517b0 con 0x563b464d1810 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce4801e040 con 0x563b464d1810 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 10 client.15630 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b46550790 con 0x563b464d1810 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce400517b0 con 0x563b464d1810 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fce48033070 con 0x563b464d1810 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 10 client.15630 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 20 client.15630 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b46550f40 0x563b46553360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 4 client.15630 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b4643c470 con 0x563b46550f40 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.461+0000 7fce577fe640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b46550f40 0x563b46553360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.462+0000 7fce377fe640 10 client.15630 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.462+0000 7fce577fe640 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b46550f40 0x563b46553360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14d07202aee813 server_cookie=cbfe298886bc3cb1 in_seq=0 out_seq=0 2024-03-18T23:12:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce377fe640 1 -- 172.21.15.132:0/480416291 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b4643c470 con 0x563b46550f40 2024-03-18T23:12:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce377fe640 10 client.15630 handle_command_reply: tid=0 2024-03-18T23:12:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 client.15630 shutdown 2024-03-18T23:12:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 msgr2=0x563b464d45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce480059f0 tx=0x7fce48009f70 comp rx=0 tx=0).stop 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 shutdown_connections 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563b46550f40 0x563b46553360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b464da5e0 0x563b464d4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b464d1810 0x563b464d45a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce377fe640 0 client.15630 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 --2- 172.21.15.132:0/480416291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563b464d0ef0 0x563b464d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.468+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 >> 172.21.15.132:0/480416291 conn(0x563b464cb810 msgr2=0x563b464d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 shutdown_connections 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce615bf740 1 -- 172.21.15.132:0/480416291 wait complete. 2024-03-18T23:12:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce615bf740 20 client.15630 trim_cache size 0 max 16384 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 msgr2=0x7fce440a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 0x7fce440a4ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce5009b960 tx=0x7fce5006d040 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 msgr2=0x7fce5819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fce5000eb10 tx=0x7fce5000c2d0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 shutdown_connections 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fce440a2a10 0x7fce440a4ed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5810e090 0x7fce5819eb20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce580744f0 0x7fce5819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 --2- 172.21.15.132:0/2865341268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58073bd0 0x7fce5819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 >> 172.21.15.132:0/2865341268 conn(0x7fce580fd2e0 msgr2=0x7fce580fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 shutdown_connections 2024-03-18T23:12:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:21.469+0000 7fce5e8f8640 1 -- 172.21.15.132:0/2865341268 wait complete. 2024-03-18T23:12:21.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:21 smithi132 ceph-mon[32020]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:21 smithi196 ceph-mon[67089]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:21 smithi144 ceph-mon[66781]: pgmap v96: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:22.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:22 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:22.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:22 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:22.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:22 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:12:23.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:23.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:23.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:12:23.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:23 smithi144 ceph-mon[66781]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:12:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:23 smithi132 ceph-mon[32020]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:23.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:12:23.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:12:23.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:12:23.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:23 smithi196 ceph-mon[67089]: pgmap v97: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:25 smithi196 ceph-mon[67089]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:25 smithi144 ceph-mon[66781]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:25 smithi132 ceph-mon[32020]: pgmap v98: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-03-18T23:12:27.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:27 smithi144 ceph-mon[66781]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:27 smithi196 ceph-mon[67089]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:27 smithi132 ceph-mon[32020]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:29 smithi196 ceph-mon[67089]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-18T23:12:29.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:29 smithi144 ceph-mon[66781]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-18T23:12:29.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:29 smithi132 ceph-mon[32020]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-03-18T23:12:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:31 smithi196 ceph-mon[67089]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:31 smithi144 ceph-mon[66781]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:31.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:31 smithi132 ceph-mon[32020]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:33 smithi196 ceph-mon[67089]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:33 smithi144 ceph-mon[66781]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:33.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:33 smithi132 ceph-mon[32020]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- 172.21.15.132:0/3104270162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 msgr2=0x7f257406a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 --2- 172.21.15.132:0/3104270162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f257406a4c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2564009900 tx=0x7f256402bc30 comp rx=0 tx=0).stop 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- 172.21.15.132:0/3104270162 shutdown_connections 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 --2- 172.21.15.132:0/3104270162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 0x7f257410e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 --2- 172.21.15.132:0/3104270162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f257406a4c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 --2- 172.21.15.132:0/3104270162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 0x7f2574107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- 172.21.15.132:0/3104270162 >> 172.21.15.132:0/3104270162 conn(0x7f2574077440 msgr2=0x7f2574077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- 172.21.15.132:0/3104270162 shutdown_connections 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- 172.21.15.132:0/3104270162 wait complete. 2024-03-18T23:12:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 Processor -- start 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.565+0000 7f257c219640 1 -- start start 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 0x7f2574198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 0x7f257419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111be0 con 0x7f257406a060 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111a60 con 0x7f25741077f0 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111d60 con 0x7f257406aa00 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52692/0 (socket says 172.21.15.132:52692) 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 -- 172.21.15.132:0/3016798050 learned_addr learned my addr 172.21.15.132:0/3016798050 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f25797b4640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 0x7f2574198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257a7b6640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 0x7f257419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 -- 172.21.15.132:0/3016798050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 msgr2=0x7f2574198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 0x7f2574198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 -- 172.21.15.132:0/3016798050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 msgr2=0x7f257419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 0x7f257419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 -- 172.21.15.132:0/3016798050 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257419cfc0 con 0x7f257406a060 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f257000ebc0 tx=0x7f257000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f256affd640 1 -- 172.21.15.132:0/3016798050 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010630 con 0x7f257406a060 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f256affd640 1 -- 172.21.15.132:0/3016798050 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2570010cb0 con 0x7f257406a060 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257419d2b0 con 0x7f257406a060 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.566+0000 7f256affd640 1 -- 172.21.15.132:0/3016798050 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570018bd0 con 0x7f257406a060 2024-03-18T23:12:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.567+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741a4b40 con 0x7f257406a060 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.569+0000 7f256affd640 1 -- 172.21.15.132:0/3016798050 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f25700107d0 con 0x7f257406a060 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.569+0000 7f257ee07740 1 Processor -- start 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.569+0000 7f257ee07740 1 -- start start 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d42ee53be0 con 0x55d42eeefef0 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d42edfc960 con 0x55d42eef0810 2024-03-18T23:12:34.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257ee07740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d42ee4fc80 con 0x55d42eef95e0 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f256affd640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 0x7f25600a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257a7b6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f25797b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257a7b6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52708/0 (socket says 172.21.15.132:52708) 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f25797b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:42214/0 (socket says 172.21.15.132:42214) 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f25797b4640 1 -- 172.21.15.132:0/1530402863 learned_addr learned my addr 172.21.15.132:0/1530402863 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:56134/0 (socket says 172.21.15.132:56134) 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f256affd640 1 -- 172.21.15.132:0/3016798050 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f25700cf3d0 con 0x7f257406a060 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f257a7b6640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 0x7f25600a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 -- 172.21.15.132:0/1530402863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 msgr2=0x55d42eefb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 -- 172.21.15.132:0/1530402863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 msgr2=0x55d42eef02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 -- 172.21.15.132:0/1530402863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42edf3d20 con 0x55d42eef0810 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f25797b4640 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.570+0000 7f2579fb5640 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25700ccb50 tx=0x7f257000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a400a235bb37557 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257a7b6640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 0x7f25600a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f256c009710 tx=0x7f256c004730 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f255ffff640 1 -- 172.21.15.132:0/1530402863 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010e60 con 0x55d42eef0810 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f255ffff640 1 -- 172.21.15.132:0/1530402863 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257009fbb0 con 0x55d42eef0810 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f255ffff640 1 -- 172.21.15.132:0/1530402863 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570098180 con 0x55d42eef0810 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 -- 172.21.15.132:0/1530402863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 msgr2=0x55d42eef90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25700ccb50 tx=0x7f257000cf90 comp rx=0 tx=0).stop 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 -- 172.21.15.132:0/1530402863 shutdown_connections 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eefb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef90a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 --2- 172.21.15.132:0/1530402863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.571+0000 7f257ee07740 1 -- 172.21.15.132:0/1530402863 >> 172.21.15.132:0/1530402863 conn(0x55d42eeea810 msgr2=0x55d42eeeabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- 172.21.15.132:0/1530402863 shutdown_connections 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- 172.21.15.132:0/1530402863 wait complete. 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- start start 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 Processor -- start 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eef6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55d42eefc1b0 con 0x55d42eeefef0 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d42ee0fdb0 con 0x55d42eef0810 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d42eefc330 con 0x55d42eef95e0 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.572+0000 7f257ee07740 10 client.0 _ll_register_callbacks cb 0x55d42ee14e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f25797b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52710/0 (socket says 172.21.15.132:52710) 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f257a7b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eef6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f25797b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:56138/0 (socket says 172.21.15.132:56138) 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 -- 172.21.15.132:0/158178825 learned_addr learned my addr 172.21.15.132:0/158178825 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 -- 172.21.15.132:0/158178825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 msgr2=0x55d42eef6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eef6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 -- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 msgr2=0x55d42eef6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 -- 172.21.15.132:0/158178825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42ee0ec60 con 0x55d42eeefef0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f2579fb5640 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25700186b0 tx=0x7f25700cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f904c928ee55f26b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f255d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010e60 con 0x55d42eeefef0 2024-03-18T23:12:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f257009b160 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f257009fbb0 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2570099e10 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f257ee07740 10 client.15642 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d42eb68210 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.573+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257009fbb0 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25700a0ad0 con 0x55d42eeefef0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 10 client.15642 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 20 client.15642 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef0b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 4 client.15642 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42ee61120 con 0x55d42ef092e0 2024-03-18T23:12:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f25797b4640 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef0b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f25797b4640 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef0b6d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9f0dec481de434 server_cookie=4ca622f12be8c1e2 in_seq=0 out_seq=0 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.574+0000 7f255d7fa640 10 client.15642 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f255d7fa640 1 -- 172.21.15.132:0/158178825 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d42ee61120 con 0x55d42ef092e0 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f255d7fa640 10 client.15642 handle_command_reply: tid=0 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 client.15642 shutdown 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 msgr2=0x55d42eef8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25700186b0 tx=0x7f25700cd000 comp rx=0 tx=0).stop 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 shutdown_connections 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef0b6d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f255d7fa640 0 client.15642 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.575+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef95e0 0x55d42eef6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42eef6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 1 --2- 172.21.15.132:0/158178825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eef8f70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 >> 172.21.15.132:0/158178825 conn(0x55d42eeea810 msgr2=0x55d42eee9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 shutdown_connections 2024-03-18T23:12:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 1 -- 172.21.15.132:0/158178825 wait complete. 2024-03-18T23:12:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.576+0000 7f257ee07740 20 client.15642 trim_cache size 0 max 16384 2024-03-18T23:12:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.590+0000 7f257ee07740 1 Processor -- start 2024-03-18T23:12:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.590+0000 7f257ee07740 1 -- start start 2024-03-18T23:12:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef0810 0x55d42eef5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef95e0 0x55d42ef0e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f25741afcb0 con 0x55d42eeefef0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25741ad580 con 0x55d42eef95e0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257ee07740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25741ab3d0 con 0x55d42eef0810 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52712/0 (socket says 172.21.15.132:52712) 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 -- 172.21.15.132:0/489860096 learned_addr learned my addr 172.21.15.132:0/489860096 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f25797b4640 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef0810 0x55d42eef5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f257a7b6640 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef95e0 0x55d42ef0e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 -- 172.21.15.132:0/489860096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef0810 msgr2=0x55d42eef5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef0810 0x55d42eef5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 -- 172.21.15.132:0/489860096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef95e0 msgr2=0x55d42ef0e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef95e0 0x55d42ef0e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 -- 172.21.15.132:0/489860096 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42eef0640 con 0x55d42eeefef0 2024-03-18T23:12:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f2579fb5640 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2570097f70 tx=0x7f257000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c778bcc2719d688 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f255f7fe640 1 -- 172.21.15.132:0/489860096 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010e60 con 0x55d42eeefef0 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f255f7fe640 1 -- 172.21.15.132:0/489860096 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2570099bb0 con 0x55d42eeefef0 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.591+0000 7f255f7fe640 1 -- 172.21.15.132:0/489860096 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257009a9d0 con 0x55d42eeefef0 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- 172.21.15.132:0/489860096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 msgr2=0x55d42eee9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2570097f70 tx=0x7f257000cf90 comp rx=0 tx=0).stop 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- 172.21.15.132:0/489860096 shutdown_connections 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef95e0 0x55d42ef0e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eef0810 0x55d42eef5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 --2- 172.21.15.132:0/489860096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eeefef0 0x55d42eee9cf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- 172.21.15.132:0/489860096 >> 172.21.15.132:0/489860096 conn(0x55d42eeea810 msgr2=0x55d42ef04e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- 172.21.15.132:0/489860096 shutdown_connections 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- 172.21.15.132:0/489860096 wait complete. 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 -- start start 2024-03-18T23:12:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.592+0000 7f257ee07740 1 Processor -- start 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 0x55d42ef0e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42ef0e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 0x55d42ef0b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f25741aba40 con 0x55d42eef95e0 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25741aceb0 con 0x55d42eef0810 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f25741ac110 con 0x55d42eeefef0 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 10 client.0 _ll_register_callbacks cb 0x7f256c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257a7b6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 0x55d42ef0b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 0x55d42ef0e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257a7b6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 0x55d42ef0b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52718/0 (socket says 172.21.15.132:52718) 2024-03-18T23:12:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257a7b6640 1 -- 172.21.15.132:0/3963612118 learned_addr learned my addr 172.21.15.132:0/3963612118 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f25797b4640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42ef0e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 -- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 msgr2=0x55d42ef0e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42ef0e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 -- 172.21.15.132:0/3963612118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 msgr2=0x55d42ef0b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 0x55d42ef0b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 -- 172.21.15.132:0/3963612118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42ec34620 con 0x55d42eeefef0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f2579fb5640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 0x55d42ef0e2a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25700186b0 tx=0x7f2570010b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08e509f9c685305 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f255d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010e60 con 0x55d42eeefef0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f25700b4d20 con 0x55d42eeefef0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.593+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d42ee5b470 con 0x55d42eeefef0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f257009b8e0 con 0x55d42eeefef0 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 10 client.25492 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d42ec34620 con 0x55d42eeefef0 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d42ee5b470 con 0x55d42eeefef0 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f25700860c0 con 0x55d42eeefef0 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 10 client.25492 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 10 client.25492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 20 client.25492 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef70dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 4 client.25492 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d42edd9770 con 0x55d42ef092e0 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.594+0000 7f25797b4640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef70dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.595+0000 7f255d7fa640 10 client.25492 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.595+0000 7f25797b4640 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef70dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=534a3a7d8f647ccf server_cookie=14b27c7351476ac5 in_seq=0 out_seq=0 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f255d7fa640 1 -- 172.21.15.132:0/3963612118 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d42edd9770 con 0x55d42ef092e0 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f255d7fa640 10 client.25492 handle_command_reply: tid=0 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 client.25492 shutdown 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 msgr2=0x55d42ef0e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 0x55d42ef0e2a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25700186b0 tx=0x7f2570010b90 comp rx=0 tx=0).stop 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 shutdown_connections 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55d42ef092e0 0x55d42ef70dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d42eef95e0 0x55d42ef0b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d42eef0810 0x55d42ef0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 --2- 172.21.15.132:0/3963612118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d42eeefef0 0x55d42ef0e2a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 >> 172.21.15.132:0/3963612118 conn(0x55d42eeea810 msgr2=0x55d42eeebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f255d7fa640 0 client.25492 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 shutdown_connections 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 1 -- 172.21.15.132:0/3963612118 wait complete. 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.601+0000 7f257ee07740 20 client.25492 trim_cache size 0 max 16384 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:12:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:12:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 msgr2=0x7f25600a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 0x7f25600a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f256c009710 tx=0x7f256c004730 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 msgr2=0x7f2574198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f257000ebc0 tx=0x7f257000c6a0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 shutdown_connections 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f25600a2a10 0x7f25600a4ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25741077f0 0x7f257419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f257406aa00 0x7f2574198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 --2- 172.21.15.132:0/3016798050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257406a060 0x7f2574198020 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 >> 172.21.15.132:0/3016798050 conn(0x7f2574077440 msgr2=0x7f25740ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 shutdown_connections 2024-03-18T23:12:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:34.602+0000 7f257c219640 1 -- 172.21.15.132:0/3016798050 wait complete. 2024-03-18T23:12:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:35 smithi196 ceph-mon[67089]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:35.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:35 smithi144 ceph-mon[66781]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:35.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:35 smithi132 ceph-mon[32020]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-03-18T23:12:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:37 smithi196 ceph-mon[67089]: pgmap v104: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-03-18T23:12:37.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:37 smithi144 ceph-mon[66781]: pgmap v104: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-03-18T23:12:37.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:37 smithi132 ceph-mon[32020]: pgmap v104: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-03-18T23:12:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:39 smithi196 ceph-mon[67089]: pgmap v105: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:39.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:39 smithi144 ceph-mon[66781]: pgmap v105: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:39.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:39 smithi132 ceph-mon[32020]: pgmap v105: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:41.668 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:41 smithi132 ceph-mon[32020]: pgmap v106: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:41 smithi196 ceph-mon[67089]: pgmap v106: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:41.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:41 smithi144 ceph-mon[66781]: pgmap v106: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:43.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:43 smithi196 ceph-mon[67089]: pgmap v107: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:43.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:43 smithi144 ceph-mon[66781]: pgmap v107: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:43.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:43 smithi132 ceph-mon[32020]: pgmap v107: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:45 smithi196 ceph-mon[67089]: pgmap v108: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:45.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:45 smithi144 ceph-mon[66781]: pgmap v108: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:45.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:45 smithi132 ceph-mon[32020]: pgmap v108: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-03-18T23:12:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 -- 172.21.15.132:0/3815307535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc4101330 msgr2=0x7f8fc410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 --2- 172.21.15.132:0/3815307535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc4101330 0x7f8fc410d550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009ca0 tx=0x7f8fbc02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 -- 172.21.15.132:0/3815307535 shutdown_connections 2024-03-18T23:12:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 --2- 172.21.15.132:0/3815307535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc4101330 0x7f8fc410d550 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 --2- 172.21.15.132:0/3815307535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 --2- 172.21.15.132:0/3815307535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc41056c0 0x7f8fc4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 -- 172.21.15.132:0/3815307535 >> 172.21.15.132:0/3815307535 conn(0x7f8fc4077580 msgr2=0x7f8fc4077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 -- 172.21.15.132:0/3815307535 shutdown_connections 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.697+0000 7f8fca95b640 1 -- 172.21.15.132:0/3815307535 wait complete. 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 Processor -- start 2024-03-18T23:12:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 -- start start 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc4101330 0x7f8fc41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc41056c0 0x7f8fc419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4111c00 con 0x7f8fc41056c0 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4111a80 con 0x7f8fc4100970 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fca95b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4111d80 con 0x7f8fc4101330 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc8ef8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc41056c0 0x7f8fc419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:58534/0 (socket says 172.21.15.132:58534) 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 -- 172.21.15.132:0/407053702 learned_addr learned my addr 172.21.15.132:0/407053702 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc4101330 0x7f8fc41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 -- 172.21.15.132:0/407053702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc4101330 msgr2=0x7f8fc41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc4101330 0x7f8fc41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 -- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc41056c0 msgr2=0x7f8fc419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc41056c0 0x7f8fc419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.698+0000 7f8fc3fff640 1 -- 172.21.15.132:0/407053702 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc419ce90 con 0x7f8fc4100970 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ea60 tx=0x7f8fac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fc17fa640 1 -- 172.21.15.132:0/407053702 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00cde0 con 0x7f8fc4100970 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fc17fa640 1 -- 172.21.15.132:0/407053702 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac0035f0 con 0x7f8fc4100970 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc419d180 con 0x7f8fc4100970 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fc17fa640 1 -- 172.21.15.132:0/407053702 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac010670 con 0x7f8fc4100970 2024-03-18T23:12:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.699+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a4a10 con 0x7f8fc4100970 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.701+0000 7f8fc17fa640 1 -- 172.21.15.132:0/407053702 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f8fac0037a0 con 0x7f8fc4100970 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.701+0000 7f8fcd623740 1 Processor -- start 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.701+0000 7f8fc17fa640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 0x7f8fb00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 -- start start 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 0x7f8fb00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc17fa640 1 -- 172.21.15.132:0/407053702 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8fac0cf120 con 0x7f8fc4100970 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x562869815be0 con 0x5628698bb5e0 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628697be960 con 0x5628698b1ef0 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fcd623740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562869811c80 con 0x5628698b2810 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:58538/0 (socket says 172.21.15.132:58538) 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 -- 172.21.15.132:0/2635761733 learned_addr learned my addr 172.21.15.132:0/2635761733 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 0x7f8fb00a4ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc004af0 tx=0x7f8fbc032360 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 -- 172.21.15.132:0/2635761733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 msgr2=0x5628698bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 -- 172.21.15.132:0/2635761733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 msgr2=0x5628698bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 -- 172.21.15.132:0/2635761733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628697b5d20 con 0x5628698b1ef0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.702+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0a3a20 tx=0x7f8fac09b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94d403d3837279b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fba7fc640 1 -- 172.21.15.132:0/2635761733 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac09f900 con 0x5628698b1ef0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fba7fc640 1 -- 172.21.15.132:0/2635761733 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac0a3f50 con 0x5628698b1ef0 2024-03-18T23:12:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fba7fc640 1 -- 172.21.15.132:0/2635761733 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0afb90 con 0x5628698b1ef0 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 -- 172.21.15.132:0/2635761733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 msgr2=0x5628698b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0a3a20 tx=0x7f8fac09b420 comp rx=0 tx=0).stop 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 -- 172.21.15.132:0/2635761733 shutdown_connections 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 --2- 172.21.15.132:0/2635761733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698b22d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 -- 172.21.15.132:0/2635761733 >> 172.21.15.132:0/2635761733 conn(0x5628698ac810 msgr2=0x5628698acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 -- 172.21.15.132:0/2635761733 shutdown_connections 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.703+0000 7f8fcd623740 1 -- 172.21.15.132:0/2635761733 wait complete. 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 -- start start 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 Processor -- start 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 0x5628698baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698bb5e0 0x5628698b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5628698be210 con 0x5628698b2810 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628697d1db0 con 0x5628698bb5e0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5628698be390 con 0x5628698b1ef0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fcd623740 10 client.0 _ll_register_callbacks cb 0x5628697d6e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 0x5628698baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38066/0 (socket says 172.21.15.132:38066) 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc37fe640 1 -- 172.21.15.132:0/4207290286 learned_addr learned my addr 172.21.15.132:0/4207290286 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698bb5e0 0x5628698b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc3fff640 1 -- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698bb5e0 msgr2=0x5628698b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698bb5e0 0x5628698b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc3fff640 1 -- 172.21.15.132:0/4207290286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 msgr2=0x5628698b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.704+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fc3fff640 1 -- 172.21.15.132:0/4207290286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628697d0c60 con 0x5628698b1ef0 2024-03-18T23:12:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 0x5628698baf80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0a3a20 tx=0x7f8fac09b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219a2e941c67d88c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac09f900 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8fac097e20 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fac0a3f50 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fac07f050 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 10 client.25504 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56286952a210 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac0a3f50 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8fac09ac70 con 0x5628698b1ef0 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 10 client.25504 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 20 client.25504 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5628698cb300 0x5628698cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 4 client.25504 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.705+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562869823120 con 0x5628698cb300 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.706+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5628698cb300 0x5628698cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.706+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5628698cb300 0x5628698cd6f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9263036bfb61a2eb server_cookie=46d4fb02f604671f in_seq=0 out_seq=0 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.706+0000 7f8fa37fe640 10 client.25504 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.706+0000 7f8fa37fe640 1 -- 172.21.15.132:0/4207290286 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562869823120 con 0x5628698cb300 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.706+0000 7f8fa37fe640 10 client.25504 handle_command_reply: tid=0 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 client.25504 shutdown 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 msgr2=0x5628698baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 0x5628698baf80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0a3a20 tx=0x7f8fac09b910 comp rx=0 tx=0).stop 2024-03-18T23:12:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 shutdown_connections 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5628698cb300 0x5628698cd6f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698bb5e0 0x5628698b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fa37fe640 0 client.25504 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 --2- 172.21.15.132:0/4207290286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b1ef0 0x5628698baf80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 >> 172.21.15.132:0/4207290286 conn(0x5628698ac810 msgr2=0x5628698abab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 shutdown_connections 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 1 -- 172.21.15.132:0/4207290286 wait complete. 2024-03-18T23:12:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.707+0000 7f8fcd623740 20 client.25504 trim_cache size 0 max 16384 2024-03-18T23:12:47.728 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:47 smithi196 ceph-mon[67089]: pgmap v109: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 Processor -- start 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 -- start start 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41ada30 con 0x5628698bb5e0 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41ab300 con 0x5628698b1ef0 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fcd623740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41a9150 con 0x5628698b2810 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38080/0 (socket says 172.21.15.132:38080) 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 -- 172.21.15.132:0/211832536 learned_addr learned my addr 172.21.15.132:0/211832536 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 -- 172.21.15.132:0/211832536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 msgr2=0x5628698baa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698baa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 -- 172.21.15.132:0/211832536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 msgr2=0x5628698d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.722+0000 7f8fc8ef8640 1 -- 172.21.15.132:0/211832536 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628698b2640 con 0x5628698bb5e0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009a60 tx=0x7f8fbc042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70dbccef19949cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/211832536 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc039040 con 0x5628698bb5e0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/211832536 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc047070 con 0x5628698bb5e0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fb9ffb640 1 -- 172.21.15.132:0/211832536 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0346a0 con 0x5628698bb5e0 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 -- 172.21.15.132:0/211832536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 msgr2=0x5628698baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009a60 tx=0x7f8fbc042040 comp rx=0 tx=0).stop 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 -- 172.21.15.132:0/211832536 shutdown_connections 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698bb5e0 0x5628698baf80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698b2810 0x5628698baa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 --2- 172.21.15.132:0/211832536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 -- 172.21.15.132:0/211832536 >> 172.21.15.132:0/211832536 conn(0x5628698ac810 msgr2=0x5628698abab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 -- 172.21.15.132:0/211832536 shutdown_connections 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.723+0000 7f8fcd623740 1 -- 172.21.15.132:0/211832536 wait complete. 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 -- start start 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 Processor -- start 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698bb5e0 0x5628698ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41a97c0 con 0x5628698b2810 2024-03-18T23:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41aac30 con 0x5628698b1ef0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41a9e90 con 0x5628698bb5e0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fcd623740 10 client.0 _ll_register_callbacks cb 0x7f8fbc045030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:38092/0 (socket says 172.21.15.132:38092) 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 -- 172.21.15.132:0/1702528886 learned_addr learned my addr 172.21.15.132:0/1702528886 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc8ef8640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698bb5e0 0x5628698ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 -- 172.21.15.132:0/1702528886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698bb5e0 msgr2=0x5628698ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698bb5e0 0x5628698ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 -- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 msgr2=0x5628698ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.724+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fc37fe640 1 -- 172.21.15.132:0/1702528886 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628695f6620 con 0x5628698b2810 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fc37fe640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5628698be4c0 tx=0x7f8fb40169a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d282fee5e4c303 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:12:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4023d30 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8fb40160d0 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc047070 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8fb40162d0 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 10 client.15666 fetch_fsmap learned FSMap version 24 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5628699316c0 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc047070 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8fb4026310 con 0x5628698b2810 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 10 client.15666 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 20 client.15666 populate_metadata read hostname 'smithi132' 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562869931df0 0x562869934210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 4 client.15666 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.725+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56286981d470 con 0x562869931df0 2024-03-18T23:12:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.726+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562869931df0 0x562869934210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:12:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.726+0000 7f8fc3fff640 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562869931df0 0x562869934210 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7629fa64a3459000 server_cookie=a94883a10a542a5f in_seq=0 out_seq=0 2024-03-18T23:12:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.726+0000 7f8fa37fe640 10 client.15666 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fa37fe640 1 -- 172.21.15.132:0/1702528886 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56286981d470 con 0x562869931df0 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fa37fe640 10 client.15666 handle_command_reply: tid=0 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 client.15666 shutdown 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 msgr2=0x5628698ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5628698be4c0 tx=0x7f8fb40169a0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 shutdown_connections 2024-03-18T23:12:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562869931df0 0x562869934210 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628698bb5e0 0x5628698ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628698b2810 0x5628698ce850 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fa37fe640 0 client.15666 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 --2- 172.21.15.132:0/1702528886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628698b1ef0 0x5628698ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 >> 172.21.15.132:0/1702528886 conn(0x5628698ac810 msgr2=0x5628698c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 shutdown_connections 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 1 -- 172.21.15.132:0/1702528886 wait complete. 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.733+0000 7f8fcd623740 20 client.15666 trim_cache size 0 max 16384 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:12:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:12:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:12:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 msgr2=0x7f8fb00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 0x7f8fb00a4ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc004af0 tx=0x7f8fbc032360 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 msgr2=0x7f8fc4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ea60 tx=0x7f8fac00ef20 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 shutdown_connections 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8fb00a2a10 0x7f8fb00a4ed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc41056c0 0x7f8fc419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fc4101330 0x7f8fc41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 --2- 172.21.15.132:0/407053702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4100970 0x7f8fc4197ea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 >> 172.21.15.132:0/407053702 conn(0x7f8fc4077580 msgr2=0x7f8fc40fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 shutdown_connections 2024-03-18T23:12:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:12:47.734+0000 7f8fca95b640 1 -- 172.21.15.132:0/407053702 wait complete. 2024-03-18T23:12:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:47 smithi144 ceph-mon[66781]: pgmap v109: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:47.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:47 smithi132 ceph-mon[32020]: pgmap v109: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-03-18T23:12:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:49 smithi196 ceph-mon[67089]: pgmap v110: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:12:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:49 smithi144 ceph-mon[66781]: pgmap v110: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:12:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:49 smithi132 ceph-mon[32020]: pgmap v110: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-03-18T23:12:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:51 smithi196 ceph-mon[67089]: pgmap v111: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:51.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:51 smithi144 ceph-mon[66781]: pgmap v111: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:51.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:51 smithi132 ceph-mon[32020]: pgmap v111: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:12:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:53 smithi196 ceph-mon[67089]: pgmap v112: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:53 smithi144 ceph-mon[66781]: pgmap v112: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:53.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:53 smithi132 ceph-mon[32020]: pgmap v112: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:55.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:55 smithi196 ceph-mon[67089]: pgmap v113: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:55 smithi144 ceph-mon[66781]: pgmap v113: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:55 smithi132 ceph-mon[32020]: pgmap v113: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:57.660 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:57 smithi144 ceph-mon[66781]: pgmap v114: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:57 smithi196 ceph-mon[67089]: pgmap v114: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:57 smithi132 ceph-mon[32020]: pgmap v114: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:12:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:12:59 smithi196 ceph-mon[67089]: pgmap v115: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:59.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:12:59 smithi144 ceph-mon[66781]: pgmap v115: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:12:59.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:12:59 smithi132 ceph-mon[32020]: pgmap v115: 97 pgs: 97 active+clean; 2.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 -- 172.21.15.132:0/3975934131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 msgr2=0x7f84ec100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 --2- 172.21.15.132:0/3975934131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec100370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009900 tx=0x7f84e402bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 -- 172.21.15.132:0/3975934131 shutdown_connections 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 --2- 172.21.15.132:0/3975934131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 --2- 172.21.15.132:0/3975934131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec100940 0x7f84ec106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 --2- 172.21.15.132:0/3975934131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec100370 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 -- 172.21.15.132:0/3975934131 >> 172.21.15.132:0/3975934131 conn(0x7f84ec076ce0 msgr2=0x7f84ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 -- 172.21.15.132:0/3975934131 shutdown_connections 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.829+0000 7f84f46bd640 1 -- 172.21.15.132:0/3975934131 wait complete. 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 Processor -- start 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 -- start start 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec19c990 0x7f84ec19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec111c00 con 0x7f84ec0fff90 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec111a80 con 0x7f84ec106b50 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f46bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec111d80 con 0x7f84ec19c990 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35052/0 (socket says 172.21.15.132:35052) 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 -- 172.21.15.132:0/2224023988 learned_addr learned my addr 172.21.15.132:0/2224023988 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f1c58640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec19c990 0x7f84ec19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 -- 172.21.15.132:0/2224023988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec19c990 msgr2=0x7f84ec19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec19c990 0x7f84ec19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 -- 172.21.15.132:0/2224023988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 msgr2=0x7f84ec198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 -- 172.21.15.132:0/2224023988 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec19d480 con 0x7f84ec0fff90 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f1c58640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.830+0000 7f84f2459640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84e4006010 tx=0x7f84e402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.831+0000 7f84e37fe640 1 -- 172.21.15.132:0/2224023988 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4041070 con 0x7f84ec0fff90 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.831+0000 7f84e37fe640 1 -- 172.21.15.132:0/2224023988 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e4004450 con 0x7f84ec0fff90 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.831+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec19d710 con 0x7f84ec0fff90 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.831+0000 7f84e37fe640 1 -- 172.21.15.132:0/2224023988 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4003e00 con 0x7f84ec0fff90 2024-03-18T23:13:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.831+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec1a4ed0 con 0x7f84ec0fff90 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.833+0000 7f84e37fe640 1 -- 172.21.15.132:0/2224023988 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f84e4044070 con 0x7f84ec0fff90 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.833+0000 7f84f7383740 1 Processor -- start 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 -- start start 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 0x5618e0d9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84e37fe640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 0x7f84d80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5618e0cf7be0 con 0x5618e0d94810 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5618e0ca0960 con 0x5618e0d9d5e0 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f7383740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5618e0cf3c80 con 0x5618e0d93ef0 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 0x5618e0d9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f1c58640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 0x7f84d80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60744/0 (socket says 172.21.15.132:60744) 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 0x5618e0d9d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35058/0 (socket says 172.21.15.132:35058) 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 -- 172.21.15.132:0/635012679 learned_addr learned my addr 172.21.15.132:0/635012679 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 -- 172.21.15.132:0/635012679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d9d5e0 msgr2=0x5618e0d9f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84e37fe640 1 -- 172.21.15.132:0/2224023988 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84e40f0780 con 0x7f84ec0fff90 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 -- 172.21.15.132:0/635012679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 msgr2=0x5618e0d9d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 0x5618e0d9d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.834+0000 7f84f2c5a640 1 -- 172.21.15.132:0/635012679 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e0c97d20 con 0x5618e0d93ef0 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.835+0000 7f84f1c58640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 0x7f84d80a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84ec104780 tx=0x7f84dc005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.835+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005c40 tx=0x7f84e8005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3ae86ce3f84a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.835+0000 7f84e0ff9640 1 -- 172.21.15.132:0/635012679 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011130 con 0x5618e0d93ef0 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.835+0000 7f84e0ff9640 1 -- 172.21.15.132:0/635012679 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e80112d0 con 0x5618e0d93ef0 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.835+0000 7f84e0ff9640 1 -- 172.21.15.132:0/635012679 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e800fb90 con 0x5618e0d93ef0 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- 172.21.15.132:0/635012679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 msgr2=0x5618e0d942d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005c40 tx=0x7f84e8005bd0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- 172.21.15.132:0/635012679 shutdown_connections 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d94810 0x5618e0d9d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- 172.21.15.132:0/635012679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0d942d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- 172.21.15.132:0/635012679 >> 172.21.15.132:0/635012679 conn(0x5618e0d8e810 msgr2=0x5618e0d8ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- 172.21.15.132:0/635012679 shutdown_connections 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- 172.21.15.132:0/635012679 wait complete. 2024-03-18T23:13:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- start start 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 Processor -- start 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5618e0da0210 con 0x5618e0d93ef0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5618e0cb3db0 con 0x5618e0d94810 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5618e0da0390 con 0x5618e0d9d5e0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f7383740 10 client.0 _ll_register_callbacks cb 0x5618e0cb8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.836+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f1c58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35064/0 (socket says 172.21.15.132:35064) 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 -- 172.21.15.132:0/339562022 learned_addr learned my addr 172.21.15.132:0/339562022 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60752/0 (socket says 172.21.15.132:60752) 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 -- 172.21.15.132:0/339562022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 msgr2=0x5618e0d9a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 -- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 msgr2=0x5618e0d9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 -- 172.21.15.132:0/339562022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e0cb2c60 con 0x5618e0d93ef0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f1c58640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f2459640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed89637ff7a508ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e404b070 con 0x5618e0d93ef0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84e40b9640 con 0x5618e0d93ef0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e80112d0 con 0x5618e0d93ef0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f84e402cd20 con 0x5618e0d93ef0 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f7383740 10 client.15678 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.837+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618e0a0c210 con 0x5618e0d93ef0 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e80112d0 con 0x5618e0d93ef0 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f84e40ccce0 con 0x5618e0d93ef0 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 20 client.15678 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0dad400 0x5618e0daf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 4 client.15678 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618e0d05120 con 0x5618e0dad400 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f1c58640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0dad400 0x5618e0daf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84d5ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.838+0000 7f84f1c58640 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0dad400 0x5618e0daf850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c552560eabd9a695 server_cookie=edb31326d894443b in_seq=0 out_seq=0 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84d5ffb640 1 -- 172.21.15.132:0/339562022 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5618e0d05120 con 0x5618e0dad400 2024-03-18T23:13:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84d5ffb640 10 client.15678 handle_command_reply: tid=0 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 client.15678 shutdown 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 msgr2=0x5618e0d9cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e40095d0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 shutdown_connections 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0dad400 0x5618e0daf850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d9a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d9a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84d5ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 --2- 172.21.15.132:0/339562022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0d9cf20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.839+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 >> 172.21.15.132:0/339562022 conn(0x5618e0d8e810 msgr2=0x5618e0d8dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.840+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 shutdown_connections 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.840+0000 7f84f7383740 1 -- 172.21.15.132:0/339562022 wait complete. 2024-03-18T23:13:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.840+0000 7f84f7383740 20 client.15678 trim_cache size 0 max 16384 2024-03-18T23:13:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 Processor -- start 2024-03-18T23:13:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 -- start start 2024-03-18T23:13:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0da8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0db5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1afd10 con 0x5618e0d9d5e0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1ad5e0 con 0x5618e0d94810 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f7383740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1ab450 con 0x5618e0d93ef0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.854+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2459640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0da8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f1c58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0db5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35068/0 (socket says 172.21.15.132:35068) 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 -- 172.21.15.132:0/607526652 learned_addr learned my addr 172.21.15.132:0/607526652 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 -- 172.21.15.132:0/607526652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 msgr2=0x5618e0da8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0da8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 -- 172.21.15.132:0/607526652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 msgr2=0x5618e0db5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0db5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 -- 172.21.15.132:0/607526652 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e0ad8620 con 0x5618e0d9d5e0 2024-03-18T23:13:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84e8011ab0 tx=0x7f84e8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8eb7cf2752bfb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84d7fff640 1 -- 172.21.15.132:0/607526652 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011040 con 0x5618e0d9d5e0 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84d7fff640 1 -- 172.21.15.132:0/607526652 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e800fda0 con 0x5618e0d9d5e0 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.855+0000 7f84d7fff640 1 -- 172.21.15.132:0/607526652 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8005580 con 0x5618e0d9d5e0 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- 172.21.15.132:0/607526652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 msgr2=0x5618e0d97570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84e8011ab0 tx=0x7f84e8009e60 comp rx=0 tx=0).stop 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- 172.21.15.132:0/607526652 shutdown_connections 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97570 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0db5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- 172.21.15.132:0/607526652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d93ef0 0x5618e0da8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- 172.21.15.132:0/607526652 >> 172.21.15.132:0/607526652 conn(0x5618e0d8e810 msgr2=0x5618e0d9a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- 172.21.15.132:0/607526652 shutdown_connections 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- 172.21.15.132:0/607526652 wait complete. 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- start start 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 Processor -- start 2024-03-18T23:13:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1abaa0 con 0x5618e0d93ef0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1acf10 con 0x5618e0d94810 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec1ac170 con 0x5618e0d9d5e0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f7383740 10 client.0 _ll_register_callbacks cb 0x7f84e4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f2459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35072/0 (socket says 172.21.15.132:35072) 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f2459640 1 -- 172.21.15.132:0/3921481090 learned_addr learned my addr 172.21.15.132:0/3921481090 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f1c58640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.856+0000 7f84f2c5a640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 -- 172.21.15.132:0/3921481090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 msgr2=0x5618e0d97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 -- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 msgr2=0x5618e0d96be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d96be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 -- 172.21.15.132:0/3921481090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e0d97950 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f2459640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b99a0 tx=0x7f84e40c5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164ea7d6329fa00a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e404b070 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84e40b7d40 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e800fda0 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f84e40c1070 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f7383740 10 client.15684 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618e0d97950 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e800fda0 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f84e40bbe90 con 0x5618e0d93ef0 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f7383740 10 client.15684 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.857+0000 7f84f7383740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f7383740 20 client.15684 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0e13a10 0x5618e0e15e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f7383740 4 client.15684 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618e0cff470 con 0x5618e0e13a10 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f1c58640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0e13a10 0x5618e0e15e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84f1c58640 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0e13a10 0x5618e0e15e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d809e77b86179e server_cookie=26180c1b5125fb2d in_seq=0 out_seq=0 2024-03-18T23:13:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.858+0000 7f84d5ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.864+0000 7f84d5ffb640 1 -- 172.21.15.132:0/3921481090 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618e0cff470 con 0x5618e0e13a10 2024-03-18T23:13:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.864+0000 7f84d5ffb640 10 client.15684 handle_command_reply: tid=0 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.864+0000 7f84f7383740 1 client.15684 shutdown 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.864+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 msgr2=0x5618e0da5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.864+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b99a0 tx=0x7f84e40c5580 comp rx=0 tx=0).stop 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 shutdown_connections 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5618e0e13a10 0x5618e0e15e00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5618e0d9d5e0 0x5618e0d97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84d5ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618e0d94810 0x5618e0d96be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 --2- 172.21.15.132:0/3921481090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5618e0d93ef0 0x5618e0da5840 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 >> 172.21.15.132:0/3921481090 conn(0x5618e0d8e810 msgr2=0x5618e0d8d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 shutdown_connections 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 1 -- 172.21.15.132:0/3921481090 wait complete. 2024-03-18T23:13:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f7383740 20 client.15684 trim_cache size 0 max 16384 2024-03-18T23:13:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:13:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:13:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:13:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 msgr2=0x7f84d80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 0x7f84d80a4ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84ec104780 tx=0x7f84dc005f90 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 msgr2=0x7f84ec198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84e4006010 tx=0x7f84e402cb30 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 shutdown_connections 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f84d80a2a10 0x7f84d80a4ed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84ec19c990 0x7f84ec19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84ec106b50 0x7f84ec198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 --2- 172.21.15.132:0/2224023988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ec0fff90 0x7f84ec198120 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 >> 172.21.15.132:0/2224023988 conn(0x7f84ec076ce0 msgr2=0x7f84ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 shutdown_connections 2024-03-18T23:13:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:00.865+0000 7f84f46bd640 1 -- 172.21.15.132:0/2224023988 wait complete. 2024-03-18T23:13:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:01 smithi196 ceph-mon[67089]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-03-18T23:13:01.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:01 smithi144 ceph-mon[66781]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-03-18T23:13:01.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:01 smithi132 ceph-mon[32020]: pgmap v116: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 39 KiB/s wr, 33 op/s 2024-03-18T23:13:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:03 smithi196 ceph-mon[67089]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:03.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:03 smithi144 ceph-mon[66781]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:03.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:03 smithi132 ceph-mon[32020]: pgmap v117: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:05.476 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.476 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:127/600: open fd = 3 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:128/600: open fd = 3 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:129/600: open fd = 3 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:130/600: open fd = 3 2024-03-18T23:13:05.477 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:131/600: open fd = 3 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:132/600: open fd = 3 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:133/600: open fd = 3 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.478 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:134/600: open fd = 3 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:135/600: open fd = 3 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:136/600: open fd = 3 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:137/600: open fd = 3 2024-03-18T23:13:05.479 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:138/600: open fd = 3 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:139/600: open fd = 3 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:140/600: open fd = 3 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:141/600: open fd = 3 2024-03-18T23:13:05.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:142/600: open fd = 3 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:143/600: open fd = 3 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:144/600: open fd = 3 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:145/600: open fd = 3 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:146/600: open fd = 3 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:147/600: open fd = 3 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.482 INFO:tasks.workunit.client.0.smithi132.stdout:148/600: open fd = 3 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:149/600: open fd = 3 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:150/600: open fd = 3 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:151/600: open fd = 3 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:152/600: open fd = 3 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:153/600: open fd = 3 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:154/600: open fd = 3 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:155/600: open fd = 3 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:156/600: open fd = 3 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:157/600: open fd = 3 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:158/600: open fd = 3 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:159/600: open fd = 3 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:160/600: open fd = 3 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:161/600: open fd = 3 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:162/600: open fd = 3 2024-03-18T23:13:05.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:163/600: open fd = 3 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:164/600: open fd = 3 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:165/600: open fd = 3 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:166/600: open fd = 3 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:167/600: open fd = 3 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:168/600: open fd = 3 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:169/600: open fd = 3 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:170/600: open fd = 3 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:171/600: open fd = 3 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:172/600: open fd = 3 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:173/600: open fd = 3 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:174/600: open fd = 3 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:175/600: open fd = 3 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:176/600: open fd = 3 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.490 INFO:tasks.workunit.client.0.smithi132.stdout:177/600: open fd = 3 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:178/600: open fd = 3 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:179/600: open fd = 3 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:180/600: open fd = 3 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.491 INFO:tasks.workunit.client.0.smithi132.stdout:181/600: open fd = 3 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:182/600: open fd = 3 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:183/600: open fd = 3 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:184/600: open fd = 3 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:185/600: open fd = 3 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:186/600: open fd = 3 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:187/600: open fd = 3 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:188/600: open fd = 3 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:13:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:05 smithi196 ceph-mon[67089]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:05 smithi144 ceph-mon[66781]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:05 smithi132 ceph-mon[32020]: pgmap v118: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:07 smithi196 ceph-mon[67089]: pgmap v119: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:13:07.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:07 smithi144 ceph-mon[66781]: pgmap v119: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:13:07.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:07 smithi132 ceph-mon[32020]: pgmap v119: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-03-18T23:13:09.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:09 smithi196 ceph-mon[67089]: pgmap v120: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:09 smithi144 ceph-mon[66781]: pgmap v120: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:09 smithi132 ceph-mon[32020]: pgmap v120: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-03-18T23:13:11.668 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:11 smithi132 ceph-mon[32020]: pgmap v121: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:13:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:11 smithi196 ceph-mon[67089]: pgmap v121: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:13:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:11 smithi144 ceph-mon[66781]: pgmap v121: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:13:13.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:13 smithi196 ceph-mon[67089]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:13 smithi144 ceph-mon[66781]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:13 smithi132 ceph-mon[32020]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 -- 172.21.15.132:0/3640204969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101006e0 msgr2=0x7ff310100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 --2- 172.21.15.132:0/3640204969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101006e0 0x7ff310100b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009900 tx=0x7ff2f802bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 -- 172.21.15.132:0/3640204969 shutdown_connections 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 --2- 172.21.15.132:0/3640204969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff31010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 --2- 172.21.15.132:0/3640204969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101006e0 0x7ff310100b60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 --2- 172.21.15.132:0/3640204969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101066f0 0x7ff310106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.960+0000 7ff315752640 1 -- 172.21.15.132:0/3640204969 >> 172.21.15.132:0/3640204969 conn(0x7ff310077840 msgr2=0x7ff310077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- 172.21.15.132:0/3640204969 shutdown_connections 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- 172.21.15.132:0/3640204969 wait complete. 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 Processor -- start 2024-03-18T23:13:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- start start 2024-03-18T23:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 0x7ff310196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 0x7ff31019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff310110b50 con 0x7ff3101010a0 2024-03-18T23:13:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101109d0 con 0x7ff3101066f0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff315752640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff310110cd0 con 0x7ff3101006e0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.961+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54332/0 (socket says 172.21.15.132:54332) 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 -- 172.21.15.132:0/4034545026 learned_addr learned my addr 172.21.15.132:0/4034545026 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30effd640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 0x7ff310196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 0x7ff31019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 -- 172.21.15.132:0/4034545026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 msgr2=0x7ff310196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 0x7ff310196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 -- 172.21.15.132:0/4034545026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 msgr2=0x7ff31019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 0x7ff31019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 -- 172.21.15.132:0/4034545026 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31019bf60 con 0x7ff3101010a0 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 0x7ff31019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30effd640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 0x7ff310196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff307fff640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802efd0 tx=0x7ff2f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30cff9640 1 -- 172.21.15.132:0/4034545026 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f802ccc0 con 0x7ff3101010a0 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31019c1f0 con 0x7ff3101010a0 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30cff9640 1 -- 172.21.15.132:0/4034545026 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f80054e0 con 0x7ff3101010a0 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff30cff9640 1 -- 172.21.15.132:0/4034545026 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f803b440 con 0x7ff3101010a0 2024-03-18T23:13:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.962+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3100699b0 con 0x7ff3101010a0 2024-03-18T23:13:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.964+0000 7ff30cff9640 1 -- 172.21.15.132:0/4034545026 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7ff2f80035f0 con 0x7ff3101010a0 2024-03-18T23:13:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 Processor -- start 2024-03-18T23:13:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 -- start start 2024-03-18T23:13:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30cff9640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 0x7ff2fc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816421810 0x55781642a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781642a5e0 0x55781642c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557816384be0 con 0x557816421810 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55781632d960 con 0x557816420ef0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff31840f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557816380c80 con 0x55781642a5e0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 0x7ff2fc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30cff9640 1 -- 172.21.15.132:0/4034545026 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff2f80f2800 con 0x7ff3101010a0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:50484/0 (socket says 172.21.15.132:50484) 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816421810 0x55781642a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.965+0000 7ff30effd640 1 -- 172.21.15.132:0/3632765254 learned_addr learned my addr 172.21.15.132:0/3632765254 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781642a5e0 0x55781642c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 -- 172.21.15.132:0/3632765254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781642a5e0 msgr2=0x55781642c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781642a5e0 0x55781642c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 -- 172.21.15.132:0/3632765254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816421810 msgr2=0x55781642a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816421810 0x55781642a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 -- 172.21.15.132:0/3632765254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557816324d20 con 0x557816420ef0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 0x7ff2fc0a4ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55781641cc30 tx=0x7ff3080113e0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff30effd640 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3000114c0 tx=0x7ff3000118c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9d7efee98362871 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff3057fa640 1 -- 172.21.15.132:0/3632765254 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300004020 con 0x557816420ef0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff3057fa640 1 -- 172.21.15.132:0/3632765254 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3000041c0 con 0x557816420ef0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.966+0000 7ff3057fa640 1 -- 172.21.15.132:0/3632765254 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30000f8e0 con 0x557816420ef0 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- 172.21.15.132:0/3632765254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 msgr2=0x5578164212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3000114c0 tx=0x7ff3000118c0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- 172.21.15.132:0/3632765254 shutdown_connections 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781642a5e0 0x55781642c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816421810 0x55781642a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 --2- 172.21.15.132:0/3632765254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x5578164212d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- 172.21.15.132:0/3632765254 >> 172.21.15.132:0/3632765254 conn(0x55781641b810 msgr2=0x55781641bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- 172.21.15.132:0/3632765254 shutdown_connections 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- 172.21.15.132:0/3632765254 wait complete. 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 -- start start 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.967+0000 7ff31840f740 1 Processor -- start 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816427330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55781642a5e0 0x557816427940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55781642d210 con 0x55781642a5e0 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557816340db0 con 0x557816420ef0 2024-03-18T23:13:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55781642d390 con 0x557816421810 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 10 client.0 _ll_register_callbacks cb 0x557816345e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff307fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816427330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:50496/0 (socket says 172.21.15.132:50496) 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55781642a5e0 0x557816427940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 -- 172.21.15.132:0/1399168282 learned_addr learned my addr 172.21.15.132:0/1399168282 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 -- 172.21.15.132:0/1399168282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 msgr2=0x557816427330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816427330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 -- 172.21.15.132:0/1399168282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55781642a5e0 msgr2=0x557816427940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55781642a5e0 0x557816427940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 -- 172.21.15.132:0/1399168282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781633fc60 con 0x557816420ef0 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff307fff640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816427330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff30effd640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff300009350 tx=0x7ff300003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f500eb384c2558 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300027020 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000041c0 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.968+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff300005040 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff300032050 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 10 client.25541 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557816099210 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3000041c0 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff300004bc0 con 0x557816420ef0 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 10 client.25541 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 20 client.25541 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781643a440 0x55781643c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 4 client.25541 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557816392120 con 0x55781643a440 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff307fff640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781643a440 0x55781643c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff2f67fc640 10 client.25541 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.969+0000 7ff307fff640 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781643a440 0x55781643c830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33179a4cd8c8b8f7 server_cookie=9e85bc73c21a43ae in_seq=0 out_seq=0 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1399168282 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557816392120 con 0x55781643a440 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff2f67fc640 10 client.25541 handle_command_reply: tid=0 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 client.25541 shutdown 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 msgr2=0x557816429fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff300009350 tx=0x7ff300003af0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 shutdown_connections 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781643a440 0x55781643c830 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55781642a5e0 0x557816427940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff2f67fc640 0 client.25541 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816427330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 --2- 172.21.15.132:0/1399168282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816420ef0 0x557816429fa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.970+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 >> 172.21.15.132:0/1399168282 conn(0x55781641b810 msgr2=0x55781641aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.971+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 shutdown_connections 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.971+0000 7ff31840f740 1 -- 172.21.15.132:0/1399168282 wait complete. 2024-03-18T23:13:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.971+0000 7ff31840f740 20 client.25541 trim_cache size 0 max 16384 2024-03-18T23:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.985+0000 7ff31840f740 1 Processor -- start 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.985+0000 7ff31840f740 1 -- start start 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 0x55781641d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816441fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101adab0 con 0x557816420ef0 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ab380 con 0x55781641b810 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff31840f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101a91f0 con 0x557816421810 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54348/0 (socket says 172.21.15.132:54348) 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 0x55781641d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816441fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 -- 172.21.15.132:0/2034110688 learned_addr learned my addr 172.21.15.132:0/2034110688 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 -- 172.21.15.132:0/2034110688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 msgr2=0x557816441fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816441fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 -- 172.21.15.132:0/2034110688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 msgr2=0x55781641d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 0x55781641d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 -- 172.21.15.132:0/2034110688 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557816421640 con 0x557816420ef0 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff30effd640 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 0x55781641d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff307fff640 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802cfd0 tx=0x7ff2f802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dadf18c7721586 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff304ff9640 1 -- 172.21.15.132:0/2034110688 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f80bf6a0 con 0x557816420ef0 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff304ff9640 1 -- 172.21.15.132:0/2034110688 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f80befa0 con 0x557816420ef0 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.986+0000 7ff304ff9640 1 -- 172.21.15.132:0/2034110688 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8087fa0 con 0x557816420ef0 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- 172.21.15.132:0/2034110688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 msgr2=0x5578164261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802cfd0 tx=0x7ff2f802bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- 172.21.15.132:0/2034110688 shutdown_connections 2024-03-18T23:13:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557816421810 0x557816441fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x5578164261e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 --2- 172.21.15.132:0/2034110688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781641b810 0x55781641d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- 172.21.15.132:0/2034110688 >> 172.21.15.132:0/2034110688 conn(0x55781641ae40 msgr2=0x557816425ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- 172.21.15.132:0/2034110688 shutdown_connections 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- 172.21.15.132:0/2034110688 wait complete. 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 -- start start 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.987+0000 7ff31840f740 1 Processor -- start 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781641b810 0x55781642e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816421810 0x55781642ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101a9840 con 0x557816420ef0 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101aacb0 con 0x557816421810 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101a9f10 con 0x55781641b810 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 10 client.0 _ll_register_callbacks cb 0x7ff308013b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54354/0 (socket says 172.21.15.132:54354) 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 -- 172.21.15.132:0/1068079882 learned_addr learned my addr 172.21.15.132:0/1068079882 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff30effd640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781641b810 0x55781642e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff30f7fe640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816421810 0x55781642ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 -- 172.21.15.132:0/1068079882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781641b810 msgr2=0x55781642e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781641b810 0x55781642e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 -- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816421810 msgr2=0x55781642ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816421810 0x55781642ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 -- 172.21.15.132:0/1068079882 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557816165620 con 0x557816420ef0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff307fff640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005ec0 tx=0x7ff2f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35246636fd00864 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f80c7740 con 0x557816420ef0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff2f80bb610 con 0x557816420ef0 2024-03-18T23:13:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f80befa0 con 0x557816420ef0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2f80bf6a0 con 0x557816420ef0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 10 client.15702 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.988+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5578164a0900 con 0x557816420ef0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f80befa0 con 0x557816420ef0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff2f8042070 con 0x557816420ef0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 10 client.15702 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 20 client.15702 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781642a5e0 0x5578164a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 4 client.15702 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55781638c470 con 0x55781642a5e0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff30effd640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781642a5e0 0x5578164a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff2f67fc640 10 client.15702 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.989+0000 7ff30effd640 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781642a5e0 0x5578164a2f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c21a1aeb7918b6 server_cookie=bb3151e4592c66c1 in_seq=0 out_seq=0 2024-03-18T23:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff2f67fc640 1 -- 172.21.15.132:0/1068079882 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55781638c470 con 0x55781642a5e0 2024-03-18T23:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff2f67fc640 10 client.15702 handle_command_reply: tid=0 2024-03-18T23:13:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 client.15702 shutdown 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 msgr2=0x55781642e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005ec0 tx=0x7ff2f8000bc0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 shutdown_connections 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55781642a5e0 0x5578164a2f30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557816421810 0x55781642ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff2f67fc640 0 client.15702 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557816420ef0 0x55781642e670 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 --2- 172.21.15.132:0/1068079882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55781641b810 0x55781642e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 >> 172.21.15.132:0/1068079882 conn(0x55781641ae40 msgr2=0x557816426360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 shutdown_connections 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.996+0000 7ff31840f740 1 -- 172.21.15.132:0/1068079882 wait complete. 2024-03-18T23:13:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff31840f740 20 client.15702 trim_cache size 0 max 16384 2024-03-18T23:13:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:13:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:13:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:13:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 msgr2=0x7ff2fc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 0x7ff2fc0a4ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55781641cc30 tx=0x7ff3080113e0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 msgr2=0x7ff3101974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802efd0 tx=0x7ff2f80042f0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 shutdown_connections 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff2fc0a2a10 0x7ff2fc0a4ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3101066f0 0x7ff31019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3101010a0 0x7ff3101974b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 --2- 172.21.15.132:0/4034545026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3101006e0 0x7ff310196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 >> 172.21.15.132:0/4034545026 conn(0x7ff310077840 msgr2=0x7ff3101032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 shutdown_connections 2024-03-18T23:13:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:13.997+0000 7ff315752640 1 -- 172.21.15.132:0/4034545026 wait complete. 2024-03-18T23:13:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:15 smithi196 ceph-mon[67089]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:15.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:15 smithi144 ceph-mon[66781]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:15.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:15 smithi132 ceph-mon[32020]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:13:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:17 smithi196 ceph-mon[67089]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:17.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:17 smithi144 ceph-mon[66781]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:17.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:17 smithi132 ceph-mon[32020]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:19 smithi196 ceph-mon[67089]: pgmap v125: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:19.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:19 smithi144 ceph-mon[66781]: pgmap v125: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:19.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:19 smithi132 ceph-mon[32020]: pgmap v125: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:21 smithi196 ceph-mon[67089]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:21.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:21 smithi144 ceph-mon[66781]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:21 smithi132 ceph-mon[32020]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:23 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:23.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:23 smithi144 ceph-mon[66781]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:23 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:23.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:23 smithi132 ceph-mon[32020]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:23 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:13:23.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:23 smithi196 ceph-mon[67089]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:24 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:24 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:24.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:24 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:13:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:24 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:24 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:24.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:24 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:13:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:24 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:13:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:24 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:13:24.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:24 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:13:25.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:25 smithi144 ceph-mon[66781]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:25.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:25 smithi132 ceph-mon[32020]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:25 smithi196 ceph-mon[67089]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 -- 172.21.15.132:0/3666856699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8100a70 msgr2=0x7fcff8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 --2- 172.21.15.132:0/3666856699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8100a70 0x7fcff8100ef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009890 tx=0x7fcfec02bb70 comp rx=0 tx=0).stop 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 -- 172.21.15.132:0/3666856699 shutdown_connections 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 --2- 172.21.15.132:0/3666856699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8101430 0x7fcff810d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 --2- 172.21.15.132:0/3666856699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8100a70 0x7fcff8100ef0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 --2- 172.21.15.132:0/3666856699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8105700 0x7fcff8105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 -- 172.21.15.132:0/3666856699 >> 172.21.15.132:0/3666856699 conn(0x7fcff8076520 msgr2=0x7fcff8076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 -- 172.21.15.132:0/3666856699 shutdown_connections 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 -- 172.21.15.132:0/3666856699 wait complete. 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.092+0000 7fcfff44d640 1 Processor -- start 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 -- start start 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 0x7fcff81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8105700 0x7fcff819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8110b30 con 0x7fcff8101430 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81109b0 con 0x7fcff8100a70 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcfff44d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8110cb0 con 0x7fcff8105700 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 0x7fcff81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35754/0 (socket says 172.21.15.132:35754) 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffc9e8640 1 -- 172.21.15.132:0/481240373 learned_addr learned my addr 172.21.15.132:0/481240373 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8105700 0x7fcff819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 -- 172.21.15.132:0/481240373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8105700 msgr2=0x7fcff819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8105700 0x7fcff819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 -- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 msgr2=0x7fcff8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 -- 172.21.15.132:0/481240373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff819d0c0 con 0x7fcff8100a70 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.093+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 0x7fcff81980d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009fd0 tx=0x7fcff000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.094+0000 7fcff67fc640 1 -- 172.21.15.132:0/481240373 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0003970 con 0x7fcff8100a70 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.094+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff819d3b0 con 0x7fcff8100a70 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.094+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a4c40 con 0x7fcff8100a70 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.094+0000 7fcff67fc640 1 -- 172.21.15.132:0/481240373 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff0004020 con 0x7fcff8100a70 2024-03-18T23:13:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.094+0000 7fcff67fc640 1 -- 172.21.15.132:0/481240373 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff000ba70 con 0x7fcff8100a70 2024-03-18T23:13:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.096+0000 7fd002031740 1 Processor -- start 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.096+0000 7fd002031740 1 -- start start 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.096+0000 7fcff67fc640 1 -- 172.21.15.132:0/481240373 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fcff00041d0 con 0x7fcff8100a70 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 0x5588834872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588834905e0 0x5588834929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5588833eabe0 con 0x558883486ef0 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558883393960 con 0x5588834905e0 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fd002031740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5588833e6c80 con 0x558883487810 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd9ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 0x5588834872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588834905e0 0x5588834929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:56724/0 (socket says 172.21.15.132:56724) 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd9ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 0x5588834872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35758/0 (socket says 172.21.15.132:35758) 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 -- 172.21.15.132:0/3003763766 learned_addr learned my addr 172.21.15.132:0/3003763766 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 -- 172.21.15.132:0/3003763766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588834905e0 msgr2=0x5588834929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588834905e0 0x5588834929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 -- 172.21.15.132:0/3003763766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 msgr2=0x5588834872d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 0x5588834872d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 -- 172.21.15.132:0/3003763766 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888338ad20 con 0x558883487810 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcff67fc640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 0x7fcfe00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cc630 tx=0x7fcff00d7580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129b272545e1d614 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 0x7fcfe00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.097+0000 7fcff67fc640 1 -- 172.21.15.132:0/481240373 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcff00cef40 con 0x7fcff8100a70 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fcfdf7fe640 1 -- 172.21.15.132:0/3003763766 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0097870 con 0x558883487810 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fcfdf7fe640 1 -- 172.21.15.132:0/3003763766 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff009bcd0 con 0x558883487810 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fcfdf7fe640 1 -- 172.21.15.132:0/3003763766 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff009af30 con 0x558883487810 2024-03-18T23:13:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 -- 172.21.15.132:0/3003763766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 msgr2=0x5588834900a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 0x7fcfe00a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009da0 tx=0x7fcfe8005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cc630 tx=0x7fcff00d7580 comp rx=0 tx=0).stop 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 -- 172.21.15.132:0/3003763766 shutdown_connections 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5588834905e0 0x5588834929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883487810 0x5588834900a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 --2- 172.21.15.132:0/3003763766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883486ef0 0x5588834872d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 -- 172.21.15.132:0/3003763766 >> 172.21.15.132:0/3003763766 conn(0x558883481810 msgr2=0x558883481bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 -- 172.21.15.132:0/3003763766 shutdown_connections 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.098+0000 7fd002031740 1 -- 172.21.15.132:0/3003763766 wait complete. 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 -- start start 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 Processor -- start 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x55888348d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x558883493090 con 0x5588834905e0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5588833a6db0 con 0x558883487810 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558883493210 con 0x558883486ef0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fd002031740 10 client.0 _ll_register_callbacks cb 0x5588833abe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fcffd1e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x55888348d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fcffd1e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:56738/0 (socket says 172.21.15.132:56738) 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fcffd9ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.099+0000 7fcffd1e9640 1 -- 172.21.15.132:0/2445838978 learned_addr learned my addr 172.21.15.132:0/2445838978 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 -- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 msgr2=0x55888348d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x55888348d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 -- 172.21.15.132:0/2445838978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 msgr2=0x55888348db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 -- 172.21.15.132:0/2445838978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588833a5c60 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cc770 tx=0x7fcff00d2650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff0b701a93ec53f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff009b110 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcff00d7d80 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff009bcd0 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcff009b3a0 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 10 client.25558 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588830ff210 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff009bcd0 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcff000b040 con 0x558883486ef0 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 10 client.25558 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 10 client.25558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 20 client.25558 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5588834a0260 0x5588834a2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 4 client.25558 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.100+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588833f8120 con 0x5588834a0260 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.101+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5588834a0260 0x5588834a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.101+0000 7fcfdcff9640 10 client.25558 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.101+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5588834a0260 0x5588834a2650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b641d4ae5ee98e6 server_cookie=e3d4344df21ec461 in_seq=0 out_seq=0 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fcfdcff9640 1 -- 172.21.15.132:0/2445838978 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5588833f8120 con 0x5588834a0260 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fcfdcff9640 10 client.25558 handle_command_reply: tid=0 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 client.25558 shutdown 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 msgr2=0x558883490080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cc770 tx=0x7fcff00d2650 comp rx=0 tx=0).stop 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 shutdown_connections 2024-03-18T23:13:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5588834a0260 0x5588834a2650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x55888348d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fcfdcff9640 0 client.25558 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 --2- 172.21.15.132:0/2445838978 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x558883490080 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 >> 172.21.15.132:0/2445838978 conn(0x558883481810 msgr2=0x558883483560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 shutdown_connections 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 1 -- 172.21.15.132:0/2445838978 wait complete. 2024-03-18T23:13:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.102+0000 7fd002031740 20 client.25558 trim_cache size 0 max 16384 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 Processor -- start 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 -- start start 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x55888348d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x5588834a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81afd90 con 0x5588834905e0 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ad660 con 0x558883487810 2024-03-18T23:13:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fd002031740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ab4b0 con 0x558883486ef0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35776/0 (socket says 172.21.15.132:35776) 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 -- 172.21.15.132:0/1068695540 learned_addr learned my addr 172.21.15.132:0/1068695540 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x55888348d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x5588834a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 -- 172.21.15.132:0/1068695540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 msgr2=0x55888348d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x55888348d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 -- 172.21.15.132:0/1068695540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 msgr2=0x5588834a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x5588834a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 -- 172.21.15.132:0/1068695540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588831cb620 con 0x5588834905e0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.117+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80058c0 tx=0x7fcfe8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc5f039d944781 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fcfdeffd640 1 -- 172.21.15.132:0/1068695540 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe801dcf0 con 0x5588834905e0 2024-03-18T23:13:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fcfdeffd640 1 -- 172.21.15.132:0/1068695540 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8005500 con 0x5588834905e0 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fcfdeffd640 1 -- 172.21.15.132:0/1068695540 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8029460 con 0x5588834905e0 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- 172.21.15.132:0/1068695540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 msgr2=0x55888348fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80058c0 tx=0x7fcfe8017000 comp rx=0 tx=0).stop 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- 172.21.15.132:0/1068695540 shutdown_connections 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588834905e0 0x55888348fb40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883487810 0x5588834a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 --2- 172.21.15.132:0/1068695540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558883486ef0 0x55888348d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- 172.21.15.132:0/1068695540 >> 172.21.15.132:0/1068695540 conn(0x558883481810 msgr2=0x558883480ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- 172.21.15.132:0/1068695540 shutdown_connections 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- 172.21.15.132:0/1068695540 wait complete. 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.118+0000 7fd002031740 1 -- start start 2024-03-18T23:13:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 Processor -- start 2024-03-18T23:13:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 0x5588834a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 0x5588834a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81abb20 con 0x558883487810 2024-03-18T23:13:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81acf90 con 0x558883486ef0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ac1f0 con 0x5588834905e0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fd002031740 10 client.0 _ll_register_callbacks cb 0x7fcfe8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35784/0 (socket says 172.21.15.132:35784) 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 -- 172.21.15.132:0/159965414 learned_addr learned my addr 172.21.15.132:0/159965414 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 0x5588834a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 0x5588834a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 -- 172.21.15.132:0/159965414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 msgr2=0x5588834a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 0x5588834a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 -- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 msgr2=0x5588834a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.119+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 0x5588834a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcffc9e8640 1 -- 172.21.15.132:0/159965414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588835054f0 con 0x558883487810 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcffd9ea640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 0x5588834a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 0x5588834a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcffc9e8640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcfec02e010 tx=0x7fcfec0041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c10b88dcb47787e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec050070 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fcfec04aab0 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe8005500 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcfec05c050 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 10 client.15720 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558883506800 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8005500 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcfec02c510 con 0x558883487810 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 10 client.15720 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 20 client.15720 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x558883506f80 0x5588835093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 4 client.15720 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.120+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588833f2470 con 0x558883506f80 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.121+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x558883506f80 0x5588835093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.121+0000 7fcffd1e9640 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x558883506f80 0x5588835093a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c94c3e9e342c534 server_cookie=95df090cf41caa08 in_seq=0 out_seq=0 2024-03-18T23:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.121+0000 7fcfdcff9640 10 client.15720 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.127+0000 7fcfdcff9640 1 -- 172.21.15.132:0/159965414 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588833f2470 con 0x558883506f80 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.127+0000 7fcfdcff9640 10 client.15720 handle_command_reply: tid=0 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 client.15720 shutdown 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 msgr2=0x5588834a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcfec02e010 tx=0x7fcfec0041c0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 shutdown_connections 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x558883506f80 0x5588835093a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5588834905e0 0x5588834a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fcfdcff9640 0 client.15720 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558883487810 0x5588834a3810 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 --2- 172.21.15.132:0/159965414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558883486ef0 0x5588834a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 >> 172.21.15.132:0/159965414 conn(0x558883481810 msgr2=0x558883480ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 shutdown_connections 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 1 -- 172.21.15.132:0/159965414 wait complete. 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.128+0000 7fd002031740 20 client.15720 trim_cache size 0 max 16384 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:13:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:13:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 msgr2=0x7fcfe00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 0x7fcfe00a4ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009da0 tx=0x7fcfe8005f90 comp rx=0 tx=0).stop 2024-03-18T23:13:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 msgr2=0x7fcff81980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 0x7fcff81980d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009fd0 tx=0x7fcff000c880 comp rx=0 tx=0).stop 2024-03-18T23:13:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 shutdown_connections 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcfe00a2a10 0x7fcfe00a4ed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcff8105700 0x7fcff819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff8101430 0x7fcff8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 --2- 172.21.15.132:0/481240373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff8100a70 0x7fcff81980d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 >> 172.21.15.132:0/481240373 conn(0x7fcff8076520 msgr2=0x7fcff806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 shutdown_connections 2024-03-18T23:13:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:27.129+0000 7fcfff44d640 1 -- 172.21.15.132:0/481240373 wait complete. 2024-03-18T23:13:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:27 smithi196 ceph-mon[67089]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:27.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:27 smithi144 ceph-mon[66781]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:27 smithi132 ceph-mon[32020]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:13:29.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:29 smithi196 ceph-mon[67089]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:29 smithi144 ceph-mon[66781]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:29.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:29 smithi132 ceph-mon[32020]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:31 smithi196 ceph-mon[67089]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:31 smithi144 ceph-mon[66781]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:31.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:31 smithi132 ceph-mon[32020]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:33 smithi196 ceph-mon[67089]: pgmap v132: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:33.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:33 smithi144 ceph-mon[66781]: pgmap v132: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:33.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:33 smithi132 ceph-mon[32020]: pgmap v132: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:35 smithi196 ceph-mon[67089]: pgmap v133: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:35.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:35 smithi144 ceph-mon[66781]: pgmap v133: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:35.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:35 smithi132 ceph-mon[32020]: pgmap v133: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:37 smithi196 ceph-mon[67089]: pgmap v134: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:37.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:37 smithi144 ceph-mon[66781]: pgmap v134: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:37.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:37 smithi132 ceph-mon[32020]: pgmap v134: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:39 smithi196 ceph-mon[67089]: pgmap v135: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:39.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:39 smithi144 ceph-mon[66781]: pgmap v135: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:39.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:39 smithi132 ceph-mon[32020]: pgmap v135: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- 172.21.15.132:0/2627608518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c1056c0 msgr2=0x7f659c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 --2- 172.21.15.132:0/2627608518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c1056c0 0x7f659c105aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:13:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- 172.21.15.132:0/2627608518 shutdown_connections 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 --2- 172.21.15.132:0/2627608518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c101330 0x7f659c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 --2- 172.21.15.132:0/2627608518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 --2- 172.21.15.132:0/2627608518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c1056c0 0x7f659c105aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- 172.21.15.132:0/2627608518 >> 172.21.15.132:0/2627608518 conn(0x7f659c077570 msgr2=0x7f659c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- 172.21.15.132:0/2627608518 shutdown_connections 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- 172.21.15.132:0/2627608518 wait complete. 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 Processor -- start 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.225+0000 7f65a23a9640 1 -- start start 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c1056c0 0x7f659c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111d00 con 0x7f659c101330 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111b80 con 0x7f659c100970 2024-03-18T23:13:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a23a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111e80 con 0x7f659c1056c0 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51850/0 (socket says 172.21.15.132:51850) 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 -- 172.21.15.132:0/4163843445 learned_addr learned my addr 172.21.15.132:0/4163843445 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659bfff640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f65a0946640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c1056c0 0x7f659c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 -- 172.21.15.132:0/4163843445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c1056c0 msgr2=0x7f659c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c1056c0 0x7f659c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 -- 172.21.15.132:0/4163843445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 msgr2=0x7f659c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659b7fe640 1 -- 172.21.15.132:0/4163843445 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c19d040 con 0x7f659c101330 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.226+0000 7f659bfff640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f659000e9f0 tx=0x7f659000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f65997fa640 1 -- 172.21.15.132:0/4163843445 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659000cd80 con 0x7f659c101330 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c19d330 con 0x7f659c101330 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f65997fa640 1 -- 172.21.15.132:0/4163843445 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6590010910 con 0x7f659c101330 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f65997fa640 1 -- 172.21.15.132:0/4163843445 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65900188f0 con 0x7f659c101330 2024-03-18T23:13:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.227+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a4bc0 con 0x7f659c101330 2024-03-18T23:13:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.229+0000 7f65997fa640 1 -- 172.21.15.132:0/4163843445 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f6590010ac0 con 0x7f659c101330 2024-03-18T23:13:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.229+0000 7f65a4f8e740 1 Processor -- start 2024-03-18T23:13:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 -- start start 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 0x56497daea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497daea5e0 0x56497daec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65997fa640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 0x7f65840a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56497da44be0 con 0x56497daea5e0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56497d9ed960 con 0x56497dae0ef0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56497da40c80 con 0x56497dae1810 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497daea5e0 0x56497daec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:53158/0 (socket says 172.21.15.132:53158) 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f659bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 0x56497daea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 -- 172.21.15.132:0/2553907079 learned_addr learned my addr 172.21.15.132:0/2553907079 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 0x7f65840a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65997fa640 1 -- 172.21.15.132:0/4163843445 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f65900cf1e0 con 0x7f659c101330 2024-03-18T23:13:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 -- 172.21.15.132:0/2553907079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 msgr2=0x56497daea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 0x56497daea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 -- 172.21.15.132:0/2553907079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497daea5e0 msgr2=0x56497daec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497daea5e0 0x56497daec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f65a0946640 1 -- 172.21.15.132:0/2553907079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497d9e4d20 con 0x56497dae0ef0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.230+0000 7f659bfff640 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 0x56497daea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65a0946640 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6594005f40 tx=0x7f659400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49935193a787e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65a0946640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 0x7f65840a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f659400fc90 tx=0x7f6594018000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65827fc640 1 -- 172.21.15.132:0/2553907079 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594022070 con 0x56497dae0ef0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65827fc640 1 -- 172.21.15.132:0/2553907079 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659400ec50 con 0x56497dae0ef0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65827fc640 1 -- 172.21.15.132:0/2553907079 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594005700 con 0x56497dae0ef0 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65a4f8e740 1 -- 172.21.15.132:0/2553907079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 msgr2=0x56497dae12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.231+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6594005f40 tx=0x7f659400f790 comp rx=0 tx=0).stop 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- 172.21.15.132:0/2553907079 shutdown_connections 2024-03-18T23:13:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497daea5e0 0x56497daec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae1810 0x56497daea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/2553907079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497dae12d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- 172.21.15.132:0/2553907079 >> 172.21.15.132:0/2553907079 conn(0x56497dadb810 msgr2=0x56497dadbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- 172.21.15.132:0/2553907079 shutdown_connections 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- 172.21.15.132:0/2553907079 wait complete. 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- start start 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 Processor -- start 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 0x56497daea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 0x56497dae79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56497daed1b0 con 0x56497dae1810 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56497da00db0 con 0x56497daea5e0 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.232+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56497daed330 con 0x56497dae0ef0 2024-03-18T23:13:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a4f8e740 10 client.0 _ll_register_callbacks cb 0x56497da05e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 0x56497daea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51878/0 (socket says 172.21.15.132:51878) 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 -- 172.21.15.132:0/4235509131 learned_addr learned my addr 172.21.15.132:0/4235509131 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a0946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 0x56497dae79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 -- 172.21.15.132:0/4235509131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 msgr2=0x56497daea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 0x56497daea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 -- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 msgr2=0x56497dae79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 0x56497dae79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 -- 172.21.15.132:0/4235509131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497d9ffc60 con 0x56497dae1810 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a0946640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 0x56497dae79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659bfff640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 0x56497daea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f659b7fe640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f659009b8b0 tx=0x7f659009b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a986d3fc7f763792 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f657b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590084e60 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f65900a3d80 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659400ec50 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f65900b9050 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a4f8e740 10 client.15732 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.233+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56497d759210 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659400ec50 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6590018b70 con 0x56497dae1810 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 10 client.15732 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 20 client.15732 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497dafa4f0 0x56497dafc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 4 client.15732 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56497da52120 con 0x56497dafa4f0 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f659bfff640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497dafa4f0 0x56497dafc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f657b7fe640 10 client.15732 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.234+0000 7f659bfff640 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497dafa4f0 0x56497dafc8e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762183904c04bab0 server_cookie=402f9b086a1930bf in_seq=0 out_seq=0 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.235+0000 7f657b7fe640 1 -- 172.21.15.132:0/4235509131 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56497da52120 con 0x56497dafa4f0 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.235+0000 7f657b7fe640 10 client.15732 handle_command_reply: tid=0 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.235+0000 7f65a4f8e740 1 client.15732 shutdown 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 msgr2=0x56497dae7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f659009b8b0 tx=0x7f659009b550 comp rx=0 tx=0).stop 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 shutdown_connections 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497dafa4f0 0x56497dafc8e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497daea5e0 0x56497dae79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497dae7390 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/4235509131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497dae0ef0 0x56497daea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f657b7fe640 0 client.15732 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 >> 172.21.15.132:0/4235509131 conn(0x56497dadb810 msgr2=0x56497dadaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 shutdown_connections 2024-03-18T23:13:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 1 -- 172.21.15.132:0/4235509131 wait complete. 2024-03-18T23:13:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.236+0000 7f65a4f8e740 20 client.15732 trim_cache size 0 max 16384 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.250+0000 7f65a4f8e740 1 Processor -- start 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.250+0000 7f65a4f8e740 1 -- start start 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497db021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497dae9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1afb80 con 0x56497dae1810 2024-03-18T23:13:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ad450 con 0x56497dae0ef0 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ab2a0 con 0x56497daea5e0 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51888/0 (socket says 172.21.15.132:51888) 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 -- 172.21.15.132:0/3364760935 learned_addr learned my addr 172.21.15.132:0/3364760935 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659bfff640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497db021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f65a0946640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497dae9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 -- 172.21.15.132:0/3364760935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 msgr2=0x56497dae9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497dae9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 -- 172.21.15.132:0/3364760935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 msgr2=0x56497db021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497db021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 -- 172.21.15.132:0/3364760935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497dae7d10 con 0x56497dae1810 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659bfff640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497db021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f659b7fe640 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f659000cfd0 tx=0x7f659009fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c2cbe224a44db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f6581ffb640 1 -- 172.21.15.132:0/3364760935 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590084e60 con 0x56497dae1810 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f6581ffb640 1 -- 172.21.15.132:0/3364760935 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65900bd0b0 con 0x56497dae1810 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.251+0000 7f6581ffb640 1 -- 172.21.15.132:0/3364760935 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659009cba0 con 0x56497dae1810 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- 172.21.15.132:0/3364760935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 msgr2=0x56497daf2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f659000cfd0 tx=0x7f659009fb40 comp rx=0 tx=0).stop 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- 172.21.15.132:0/3364760935 shutdown_connections 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497dae9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae1810 0x56497daf2570 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/3364760935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae0ef0 0x56497db021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- 172.21.15.132:0/3364760935 >> 172.21.15.132:0/3364760935 conn(0x56497dadb810 msgr2=0x56497dadcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- 172.21.15.132:0/3364760935 shutdown_connections 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- 172.21.15.132:0/3364760935 wait complete. 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 -- start start 2024-03-18T23:13:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.252+0000 7f65a4f8e740 1 Processor -- start 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 0x56497dae47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497db59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ab910 con 0x56497dae0ef0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1acd80 con 0x56497dae1810 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1abfe0 con 0x56497daea5e0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 10 client.0 _ll_register_callbacks cb 0x7f659000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 0x56497dae47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:53176/0 (socket says 172.21.15.132:53176) 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659b7fe640 1 -- 172.21.15.132:0/631437032 learned_addr learned my addr 172.21.15.132:0/631437032 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a0946640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497db59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 -- 172.21.15.132:0/631437032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 msgr2=0x56497db59830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497db59830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 -- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 msgr2=0x56497dae4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 -- 172.21.15.132:0/631437032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497daecd40 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659b7fe640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a0946640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497db59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f659bfff640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 0x56497dae47c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f658c000c00 tx=0x7f658c0040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b5601d149d4c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f657b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c02c8b0 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f658c02ca50 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.253+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56497da4c470 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f658c05a050 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 10 client.15738 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56497d825620 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56497da4c470 con 0x56497dae0ef0 2024-03-18T23:13:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f658c04b360 con 0x56497dae0ef0 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 10 client.15738 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 20 client.15738 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497db615d0 0x56497db639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 4 client.15738 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56497d9ca770 con 0x56497db615d0 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.254+0000 7f659b7fe640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497db615d0 0x56497db639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.255+0000 7f657b7fe640 10 client.15738 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.255+0000 7f659b7fe640 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497db615d0 0x56497db639f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba3c5c08b896b9e server_cookie=171d19af928e9497 in_seq=0 out_seq=0 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f657b7fe640 1 -- 172.21.15.132:0/631437032 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56497d9ca770 con 0x56497db615d0 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f657b7fe640 10 client.15738 handle_command_reply: tid=0 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 client.15738 shutdown 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 msgr2=0x56497dae47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 0x56497dae47c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f658c000c00 tx=0x7f658c0040d0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 shutdown_connections 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56497db615d0 0x56497db639f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56497daea5e0 0x56497db59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f657b7fe640 0 client.15738 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56497dae1810 0x56497dae4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 --2- 172.21.15.132:0/631437032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56497dae0ef0 0x56497dae47c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 >> 172.21.15.132:0/631437032 conn(0x56497dadb810 msgr2=0x56497dadd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 shutdown_connections 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 1 -- 172.21.15.132:0/631437032 wait complete. 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.262+0000 7f65a4f8e740 20 client.15738 trim_cache size 0 max 16384 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:13:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:13:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 msgr2=0x7f65840a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 0x7f65840a4ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f659400fc90 tx=0x7f6594018000 comp rx=0 tx=0).stop 2024-03-18T23:13:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 msgr2=0x7f659c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f659000e9f0 tx=0x7f659000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 shutdown_connections 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f65840a2a10 0x7f65840a4ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f659c1056c0 0x7f659c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f659c101330 0x7f659c1985e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 --2- 172.21.15.132:0/4163843445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f659c100970 0x7f659c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 >> 172.21.15.132:0/4163843445 conn(0x7f659c077570 msgr2=0x7f659c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 shutdown_connections 2024-03-18T23:13:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:40.263+0000 7f65a23a9640 1 -- 172.21.15.132:0/4163843445 wait complete. 2024-03-18T23:13:41.668 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:41 smithi132 ceph-mon[32020]: pgmap v136: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:13:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:41 smithi196 ceph-mon[67089]: pgmap v136: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:13:41.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:41 smithi144 ceph-mon[66781]: pgmap v136: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:13:43.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:43 smithi196 ceph-mon[67089]: pgmap v137: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:43 smithi144 ceph-mon[66781]: pgmap v137: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:43.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:43 smithi132 ceph-mon[32020]: pgmap v137: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:45 smithi196 ceph-mon[67089]: pgmap v138: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:45.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:45 smithi144 ceph-mon[66781]: pgmap v138: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:45.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:45 smithi132 ceph-mon[32020]: pgmap v138: 97 pgs: 97 active+clean; 2.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-03-18T23:13:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:47 smithi196 ceph-mon[67089]: pgmap v139: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-03-18T23:13:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:47 smithi144 ceph-mon[66781]: pgmap v139: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-03-18T23:13:47.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:47 smithi132 ceph-mon[32020]: pgmap v139: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-03-18T23:13:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:49 smithi196 ceph-mon[67089]: pgmap v140: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:49.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:49 smithi144 ceph-mon[66781]: pgmap v140: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:49 smithi132 ceph-mon[32020]: pgmap v140: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:51 smithi196 ceph-mon[67089]: pgmap v141: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:51 smithi144 ceph-mon[66781]: pgmap v141: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:51.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:51 smithi132 ceph-mon[32020]: pgmap v141: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- 172.21.15.132:0/3242676581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc601006e0 msgr2=0x7fdc6010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 --2- 172.21.15.132:0/3242676581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc601006e0 0x7fdc6010e6b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009cd0 tx=0x7fdc5802c0f0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- 172.21.15.132:0/3242676581 shutdown_connections 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 --2- 172.21.15.132:0/3242676581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc601006e0 0x7fdc6010e6b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 --2- 172.21.15.132:0/3242676581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc600ffd20 0x7fdc601001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 --2- 172.21.15.132:0/3242676581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc60106800 0x7fdc60106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- 172.21.15.132:0/3242676581 >> 172.21.15.132:0/3242676581 conn(0x7fdc60076ce0 msgr2=0x7fdc600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- 172.21.15.132:0/3242676581 shutdown_connections 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- 172.21.15.132:0/3242676581 wait complete. 2024-03-18T23:13:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 Processor -- start 2024-03-18T23:13:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.359+0000 7fdc659c4640 1 -- start start 2024-03-18T23:13:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc601006e0 0x7fdc60198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 0x7fdc6019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111cd0 con 0x7fdc60106800 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111b50 con 0x7fdc600ffd20 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc659c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111e50 con 0x7fdc601006e0 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 0x7fdc6019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:59068/0 (socket says 172.21.15.132:59068) 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 0x7fdc6019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33860/0 (socket says 172.21.15.132:33860) 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc567fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc601006e0 0x7fdc60198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 -- 172.21.15.132:0/916361195 learned_addr learned my addr 172.21.15.132:0/916361195 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 -- 172.21.15.132:0/916361195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc601006e0 msgr2=0x7fdc60198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc601006e0 0x7fdc60198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 -- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 msgr2=0x7fdc6019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 0x7fdc6019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.360+0000 7fdc5effd640 1 -- 172.21.15.132:0/916361195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6019d070 con 0x7fdc600ffd20 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc5effd640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9f0 tx=0x7fdc5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc649c2640 1 -- 172.21.15.132:0/916361195 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000cd80 con 0x7fdc600ffd20 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6019d360 con 0x7fdc600ffd20 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc649c2640 1 -- 172.21.15.132:0/916361195 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc50010910 con 0x7fdc600ffd20 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc649c2640 1 -- 172.21.15.132:0/916361195 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500188f0 con 0x7fdc600ffd20 2024-03-18T23:13:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.361+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a4bf0 con 0x7fdc600ffd20 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.363+0000 7fdc649c2640 1 -- 172.21.15.132:0/916361195 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fdc50010430 con 0x7fdc600ffd20 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.363+0000 7fdc649c2640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 0x7fdc4c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.363+0000 7fdc567fc640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 0x7fdc4c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.363+0000 7fdc649c2640 1 -- 172.21.15.132:0/916361195 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fdc500cf090 con 0x7fdc600ffd20 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 Processor -- start 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 -- start start 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc567fc640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 0x7fdc4c0a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc60106070 tx=0x7fdc48005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0e62b0 0x5565ee0f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0f39f0 0x5565ee0f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5565ee049be0 con 0x5565ee0f39f0 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565edff2960 con 0x5565ee0e5990 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc685ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565ee045c80 con 0x5565ee0e62b0 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc567fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0e62b0 0x5565ee0f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0f39f0 0x5565ee0f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:59084/0 (socket says 172.21.15.132:59084) 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 -- 172.21.15.132:0/4095239435 learned_addr learned my addr 172.21.15.132:0/4095239435 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 -- 172.21.15.132:0/4095239435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0e62b0 msgr2=0x5565ee0f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0e62b0 0x5565ee0f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 -- 172.21.15.132:0/4095239435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0f39f0 msgr2=0x5565ee0f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0f39f0 0x5565ee0f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.364+0000 7fdc5effd640 1 -- 172.21.15.132:0/4095239435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565edfe9d20 con 0x5565ee0e5990 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9c0 tx=0x7fdc5009b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a774d5f332a651 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc55ffb640 1 -- 172.21.15.132:0/4095239435 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50010720 con 0x5565ee0e5990 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc55ffb640 1 -- 172.21.15.132:0/4095239435 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc500ccb30 con 0x5565ee0e5990 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc55ffb640 1 -- 172.21.15.132:0/4095239435 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5009acc0 con 0x5565ee0e5990 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 -- 172.21.15.132:0/4095239435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 msgr2=0x5565ee0e5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9c0 tx=0x7fdc5009b9b0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 -- 172.21.15.132:0/4095239435 shutdown_connections 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0f39f0 0x5565ee0f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0e62b0 0x5565ee0f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4095239435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e5990 0x5565ee0e5d70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 -- 172.21.15.132:0/4095239435 >> 172.21.15.132:0/4095239435 conn(0x5565ee0e0810 msgr2=0x5565ee0e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 -- 172.21.15.132:0/4095239435 shutdown_connections 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.365+0000 7fdc685ad740 1 -- 172.21.15.132:0/4095239435 wait complete. 2024-03-18T23:13:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 -- start start 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 Processor -- start 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5565ee0f6620 con 0x5565ee0e5990 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565ee005db0 con 0x5565ee0e62b0 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565ee0f67a0 con 0x5565ee0f39f0 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc685ad740 10 client.0 _ll_register_callbacks cb 0x5565ee00ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33884/0 (socket says 172.21.15.132:33884) 2024-03-18T23:13:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc5effd640 1 -- 172.21.15.132:0/4151334778 learned_addr learned my addr 172.21.15.132:0/4151334778 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.366+0000 7fdc567fc640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 -- 172.21.15.132:0/4151334778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 msgr2=0x5565ee0f0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 -- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 msgr2=0x5565ee0f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 -- 172.21.15.132:0/4151334778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ee004c60 con 0x5565ee0e5990 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5f7fe640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc5effd640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc500a3a30 tx=0x7fdc5000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b424708a38222d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50010720 con 0x5565ee0e5990 2024-03-18T23:13:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fdc500afba0 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc500ccb30 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc500afda0 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc685ad740 10 client.15750 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5565edd5e210 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc500ccb30 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.367+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdc500977c0 con 0x5565ee0e5990 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 10 client.15750 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 20 client.15750 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee1037d0 0x5565ee105bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 4 client.15750 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565ee057120 con 0x5565ee1037d0 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc567fc640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee1037d0 0x5565ee105bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc3effd640 10 client.15750 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.368+0000 7fdc567fc640 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee1037d0 0x5565ee105bc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30b81b8e4b89372 server_cookie=53340442ab389eaa in_seq=0 out_seq=0 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc3effd640 1 -- 172.21.15.132:0/4151334778 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5565ee057120 con 0x5565ee1037d0 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc3effd640 10 client.15750 handle_command_reply: tid=0 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 client.15750 shutdown 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 msgr2=0x5565ee0f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc500a3a30 tx=0x7fdc5000c840 comp rx=0 tx=0).stop 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 shutdown_connections 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee1037d0 0x5565ee105bc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc3effd640 0 client.15750 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 --2- 172.21.15.132:0/4151334778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee0f33f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 >> 172.21.15.132:0/4151334778 conn(0x5565ee0e0810 msgr2=0x5565ee0e2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 shutdown_connections 2024-03-18T23:13:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 1 -- 172.21.15.132:0/4151334778 wait complete. 2024-03-18T23:13:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.369+0000 7fdc685ad740 20 client.15750 trim_cache size 0 max 16384 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 Processor -- start 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 -- start start 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601ada80 con 0x5565ee0e5990 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601ab350 con 0x5565ee0e62b0 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.384+0000 7fdc685ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601a91c0 con 0x5565ee0f39f0 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33898/0 (socket says 172.21.15.132:33898) 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 -- 172.21.15.132:0/865162911 learned_addr learned my addr 172.21.15.132:0/865162911 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5f7fe640 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc567fc640 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 -- 172.21.15.132:0/865162911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 msgr2=0x5565ee0f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 -- 172.21.15.132:0/865162911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 msgr2=0x5565ee0f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 -- 172.21.15.132:0/865162911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ee0f1100 con 0x5565ee0e5990 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc5effd640 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc50099b20 tx=0x7fdc50097b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2519e68efa787bb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc557fa640 1 -- 172.21.15.132:0/865162911 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ccb30 con 0x5565ee0e5990 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc557fa640 1 -- 172.21.15.132:0/865162911 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5009a9a0 con 0x5565ee0e5990 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.385+0000 7fdc557fa640 1 -- 172.21.15.132:0/865162911 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500975c0 con 0x5565ee0e5990 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- 172.21.15.132:0/865162911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 msgr2=0x5565ee108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc50099b20 tx=0x7fdc50097b90 comp rx=0 tx=0).stop 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- 172.21.15.132:0/865162911 shutdown_connections 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee0f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee0f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 --2- 172.21.15.132:0/865162911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108be0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- 172.21.15.132:0/865162911 >> 172.21.15.132:0/865162911 conn(0x5565ee0e0810 msgr2=0x5565ee0e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- 172.21.15.132:0/865162911 shutdown_connections 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- 172.21.15.132:0/865162911 wait complete. 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 -- start start 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.386+0000 7fdc685ad740 1 Processor -- start 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee109010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601a9810 con 0x5565ee0e5990 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601aac80 con 0x5565ee0e62b0 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601a9ee0 con 0x5565ee0f39f0 2024-03-18T23:13:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 10 client.0 _ll_register_callbacks cb 0x7fdc50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:33908/0 (socket says 172.21.15.132:33908) 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 -- 172.21.15.132:0/1157221699 learned_addr learned my addr 172.21.15.132:0/1157221699 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5f7fe640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc567fc640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 -- 172.21.15.132:0/1157221699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 msgr2=0x5565ee107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 -- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 msgr2=0x5565ee109010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee109010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 -- 172.21.15.132:0/1157221699 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ede2a620 con 0x5565ee0e5990 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc5effd640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc5009a920 tx=0x7fdc5009abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a12678cb450db38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ae060 con 0x5565ee0e5990 2024-03-18T23:13:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fdc50099b70 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.387+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5009a9a0 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc50010720 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 10 client.15756 fetch_fsmap learned FSMap version 24 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5565ede2a620 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5009a9a0 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdc500a0830 con 0x5565ee0e5990 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 10 client.15756 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 20 client.15756 populate_metadata read hostname 'smithi132' 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee16a3a0 0x5565ee16c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 4 client.15756 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565ee051470 con 0x5565ee16a3a0 2024-03-18T23:13:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.388+0000 7fdc567fc640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee16a3a0 0x5565ee16c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:13:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.389+0000 7fdc3effd640 10 client.15756 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.389+0000 7fdc567fc640 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee16a3a0 0x5565ee16c790 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cf4c49ebf6a6e9 server_cookie=35cb8bfbb5792553 in_seq=0 out_seq=0 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc3effd640 1 -- 172.21.15.132:0/1157221699 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565ee051470 con 0x5565ee16a3a0 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc3effd640 10 client.15756 handle_command_reply: tid=0 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 client.15756 shutdown 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 msgr2=0x5565ee108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc5009a920 tx=0x7fdc5009abd0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 shutdown_connections 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5565ee16a3a0 0x5565ee16c790 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565ee0f39f0 0x5565ee107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565ee0e62b0 0x5565ee109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc3effd640 0 client.15756 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:13:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 --2- 172.21.15.132:0/1157221699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565ee0e5990 0x5565ee108ad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 >> 172.21.15.132:0/1157221699 conn(0x5565ee0e0810 msgr2=0x5565ee0e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 shutdown_connections 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.395+0000 7fdc685ad740 1 -- 172.21.15.132:0/1157221699 wait complete. 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc685ad740 20 client.15756 trim_cache size 0 max 16384 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:13:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:13:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 msgr2=0x7fdc4c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 0x7fdc4c0a4ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc60106070 tx=0x7fdc48005f90 comp rx=0 tx=0).stop 2024-03-18T23:13:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 msgr2=0x7fdc601980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:13:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9f0 tx=0x7fdc5000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 shutdown_connections 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdc4c0a2a10 0x7fdc4c0a4ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc60106800 0x7fdc6019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc601006e0 0x7fdc60198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 --2- 172.21.15.132:0/916361195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc600ffd20 0x7fdc601980d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 >> 172.21.15.132:0/916361195 conn(0x7fdc60076ce0 msgr2=0x7fdc60104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 shutdown_connections 2024-03-18T23:13:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:13:53.396+0000 7fdc659c4640 1 -- 172.21.15.132:0/916361195 wait complete. 2024-03-18T23:13:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:53 smithi196 ceph-mon[67089]: pgmap v142: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:53.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:53 smithi144 ceph-mon[66781]: pgmap v142: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:53.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:53 smithi132 ceph-mon[32020]: pgmap v142: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:55 smithi196 ceph-mon[67089]: pgmap v143: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:55.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:55 smithi144 ceph-mon[66781]: pgmap v143: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:55 smithi132 ceph-mon[32020]: pgmap v143: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:13:57.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:57 smithi144 ceph-mon[66781]: pgmap v144: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:57 smithi196 ceph-mon[67089]: pgmap v144: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:57 smithi132 ceph-mon[32020]: pgmap v144: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T23:13:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:13:59 smithi196 ceph-mon[67089]: pgmap v145: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:59.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:13:59 smithi144 ceph-mon[66781]: pgmap v145: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:13:59.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:13:59 smithi132 ceph-mon[32020]: pgmap v145: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:01 smithi196 ceph-mon[67089]: pgmap v146: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:01.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:01 smithi144 ceph-mon[66781]: pgmap v146: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:01.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:01 smithi132 ceph-mon[32020]: pgmap v146: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:03 smithi196 ceph-mon[67089]: pgmap v147: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:03.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:03 smithi144 ceph-mon[66781]: pgmap v147: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:03.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:03 smithi132 ceph-mon[32020]: pgmap v147: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:05 smithi196 ceph-mon[67089]: pgmap v148: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:05.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:05 smithi144 ceph-mon[66781]: pgmap v148: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:05 smithi132 ceph-mon[32020]: pgmap v148: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-03-18T23:14:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- 172.21.15.132:0/1547314180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c100970 msgr2=0x7f6c6c100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1547314180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c100970 0x7f6c6c100dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009900 tx=0x7f6c5c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- 172.21.15.132:0/1547314180 shutdown_connections 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1547314180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1547314180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c100970 0x7f6c6c100dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1547314180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c1056a0 0x7f6c6c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- 172.21.15.132:0/1547314180 >> 172.21.15.132:0/1547314180 conn(0x7f6c6c0774f0 msgr2=0x7f6c6c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- 172.21.15.132:0/1547314180 shutdown_connections 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- 172.21.15.132:0/1547314180 wait complete. 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 Processor -- start 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 -- start start 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.492+0000 7f6c7417e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c0fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c1056a0 0x7f6c6c0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c110bb0 con 0x7f6c6c100970 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c110a30 con 0x7f6c6c1056a0 2024-03-18T23:14:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c110d30 con 0x7f6c6c101310 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60182/0 (socket says 172.21.15.132:60182) 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 -- 172.21.15.132:0/1584450007 learned_addr learned my addr 172.21.15.132:0/1584450007 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c1056a0 0x7f6c6c0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71719640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c0fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 -- 172.21.15.132:0/1584450007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 msgr2=0x7f6c6c0fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c0fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 -- 172.21.15.132:0/1584450007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c1056a0 msgr2=0x7f6c6c0feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c1056a0 0x7f6c6c0feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 -- 172.21.15.132:0/1584450007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6c1a65a0 con 0x7f6c6c100970 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71719640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c0fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800ebc0 tx=0x7f6c6800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c62ffd640 1 -- 172.21.15.132:0/1584450007 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010630 con 0x7f6c6c100970 2024-03-18T23:14:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6c1a6890 con 0x7f6c6c100970 2024-03-18T23:14:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c62ffd640 1 -- 172.21.15.132:0/1584450007 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68010cb0 con 0x7f6c6c100970 2024-03-18T23:14:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c62ffd640 1 -- 172.21.15.132:0/1584450007 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68018bd0 con 0x7f6c6c100970 2024-03-18T23:14:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.493+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6c1a6e00 con 0x7f6c6c100970 2024-03-18T23:14:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.496+0000 7f6c62ffd640 1 -- 172.21.15.132:0/1584450007 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f6c680107d0 con 0x7f6c6c100970 2024-03-18T23:14:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.496+0000 7f6c76d69740 1 Processor -- start 2024-03-18T23:14:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.496+0000 7f6c76d69740 1 -- start start 2024-03-18T23:14:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.496+0000 7f6c62ffd640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 0x7f6c580a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cd42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cdd5e0 0x563736cdf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563736c37be0 con 0x563736cd3ef0 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563736be0960 con 0x563736cd4810 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c76d69740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563736c33c80 con 0x563736cdd5e0 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 0x7f6c580a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cd42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:34910/0 (socket says 172.21.15.132:34910) 2024-03-18T23:14:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cd42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60196/0 (socket says 172.21.15.132:60196) 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 -- 172.21.15.132:0/1704953451 learned_addr learned my addr 172.21.15.132:0/1704953451 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cdd5e0 0x563736cdf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c62ffd640 1 -- 172.21.15.132:0/1584450007 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6c680cf390 con 0x7f6c6c100970 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 -- 172.21.15.132:0/1704953451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cdd5e0 msgr2=0x563736cdf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cdd5e0 0x563736cdf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 -- 172.21.15.132:0/1704953451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 msgr2=0x563736cd42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cd42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 -- 172.21.15.132:0/1704953451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563736bd7d20 con 0x563736cd4810 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c71719640 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c02c170 tx=0x7f6c5c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55dc8296289221ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 0x7f6c580a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009710 tx=0x7f6c64009ad0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c57fff640 1 -- 172.21.15.132:0/1704953451 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c042070 con 0x563736cd4810 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c57fff640 1 -- 172.21.15.132:0/1704953451 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c004a70 con 0x563736cd4810 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.497+0000 7f6c57fff640 1 -- 172.21.15.132:0/1704953451 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c005160 con 0x563736cd4810 2024-03-18T23:14:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 -- 172.21.15.132:0/1704953451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 msgr2=0x563736cdd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c02c170 tx=0x7f6c5c009560 comp rx=0 tx=0).stop 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 -- 172.21.15.132:0/1704953451 shutdown_connections 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cdd5e0 0x563736cdf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd4810 0x563736cdd0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1704953451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cd42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 -- 172.21.15.132:0/1704953451 >> 172.21.15.132:0/1704953451 conn(0x563736cce810 msgr2=0x563736ccebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 -- 172.21.15.132:0/1704953451 shutdown_connections 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.498+0000 7f6c76d69740 1 -- 172.21.15.132:0/1704953451 wait complete. 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 -- start start 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 Processor -- start 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cda300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736cda880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563736ce01b0 con 0x563736cd3ef0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563736bf3db0 con 0x563736cdd5e0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563736ce0330 con 0x563736cd4810 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c76d69740 10 client.0 _ll_register_callbacks cb 0x563736bf8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60204/0 (socket says 172.21.15.132:60204) 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c71f1a640 1 -- 172.21.15.132:0/4111966209 learned_addr learned my addr 172.21.15.132:0/4111966209 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c7271b640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736cda880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.499+0000 7f6c71719640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cda300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 -- 172.21.15.132:0/4111966209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 msgr2=0x563736cda300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cda300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 -- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 msgr2=0x563736cda880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736cda880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 -- 172.21.15.132:0/4111966209 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563736bf2c60 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c7271b640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736cda880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6c6809ba60 tx=0x7f6c6801da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f423c0aee0e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010e60 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6c680b4ea0 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c5c004a70 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c68026040 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c76d69740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56373694c210 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c004a70 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6c680b8fc0 con 0x563736cd3ef0 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c76d69740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.500+0000 7f6c76d69740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c76d69740 20 client.15768 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736ced3e0 0x563736cef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c76d69740 4 client.15768 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563736c45120 con 0x563736ced3e0 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c71719640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736ced3e0 0x563736cef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c557fa640 10 client.15768 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.501+0000 7f6c71719640 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736ced3e0 0x563736cef7d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebdbb9c61d97c94d server_cookie=b61880142ca24b67 in_seq=0 out_seq=0 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c557fa640 1 -- 172.21.15.132:0/4111966209 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563736c45120 con 0x563736ced3e0 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c557fa640 10 client.15768 handle_command_reply: tid=0 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 client.15768 shutdown 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 msgr2=0x563736cdcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6c6809ba60 tx=0x7f6c6801da00 comp rx=0 tx=0).stop 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 shutdown_connections 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736ced3e0 0x563736cef7d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736cda880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cda300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 --2- 172.21.15.132:0/4111966209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736cdcf00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c557fa640 0 client.15768 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 >> 172.21.15.132:0/4111966209 conn(0x563736cce810 msgr2=0x563736ccdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 shutdown_connections 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 1 -- 172.21.15.132:0/4111966209 wait complete. 2024-03-18T23:14:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.502+0000 7f6c76d69740 20 client.15768 trim_cache size 0 max 16384 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 Processor -- start 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 -- start start 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 0x563736ce8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cf5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1b1ec0 con 0x563736cdd5e0 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1af790 con 0x563736cd3ef0 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c76d69740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1ad5e0 con 0x563736cd4810 2024-03-18T23:14:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60212/0 (socket says 172.21.15.132:60212) 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 -- 172.21.15.132:0/1341758632 learned_addr learned my addr 172.21.15.132:0/1341758632 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 0x563736ce8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c71719640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cf5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 -- 172.21.15.132:0/1341758632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 msgr2=0x563736cf5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cf5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 -- 172.21.15.132:0/1341758632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 msgr2=0x563736ce8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 0x563736ce8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 -- 172.21.15.132:0/1341758632 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563736cdabf0 con 0x563736cdd5e0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 0x563736ce8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.517+0000 7f6c7271b640 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009e80 tx=0x7f6c64021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da7cf7211ab05ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c577fe640 1 -- 172.21.15.132:0/1341758632 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64013e60 con 0x563736cdd5e0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c577fe640 1 -- 172.21.15.132:0/1341758632 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640044b0 con 0x563736cdd5e0 2024-03-18T23:14:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c577fe640 1 -- 172.21.15.132:0/1341758632 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004ce0 con 0x563736cdd5e0 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 -- 172.21.15.132:0/1341758632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 msgr2=0x563736cdca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009e80 tx=0x7f6c64021000 comp rx=0 tx=0).stop 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 -- 172.21.15.132:0/1341758632 shutdown_connections 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cdd5e0 0x563736cdca30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736cf5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 --2- 172.21.15.132:0/1341758632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cd3ef0 0x563736ce8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 -- 172.21.15.132:0/1341758632 >> 172.21.15.132:0/1341758632 conn(0x563736cce810 msgr2=0x563736ccebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 -- 172.21.15.132:0/1341758632 shutdown_connections 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.518+0000 7f6c76d69740 1 -- 172.21.15.132:0/1341758632 wait complete. 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 -- start start 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 Processor -- start 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736ce5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736ce0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736ce10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1adc50 con 0x563736cd3ef0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1af0c0 con 0x563736cdd5e0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1ae320 con 0x563736cd4810 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c76d69740 10 client.0 _ll_register_callbacks cb 0x7f6c6801cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736ce5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c71f1a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736ce5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60216/0 (socket says 172.21.15.132:60216) 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736ce10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c71f1a640 1 -- 172.21.15.132:0/3617912312 learned_addr learned my addr 172.21.15.132:0/3617912312 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c71719640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736ce0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 -- 172.21.15.132:0/3617912312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 msgr2=0x563736ce0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736ce0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 -- 172.21.15.132:0/3617912312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 msgr2=0x563736ce5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736ce5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.519+0000 7f6c7271b640 1 -- 172.21.15.132:0/3617912312 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563736a18620 con 0x563736cdd5e0 2024-03-18T23:14:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c7271b640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736ce10a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d10 tx=0x7f6c64005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480dcb03e2e2125 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64013e60 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c640044b0 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6c64033450 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c64033650 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 10 client.25619 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563736d53420 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640044b0 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6c64020050 con 0x563736cdd5e0 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 10 client.25619 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 20 client.25619 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736d53ac0 0x563736d55ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 4 client.25619 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563736c3f470 con 0x563736d53ac0 2024-03-18T23:14:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.520+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736d53ac0 0x563736d55ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.521+0000 7f6c557fa640 10 client.25619 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.521+0000 7f6c71f1a640 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736d53ac0 0x563736d55ee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc6d9bc5ce6c62f server_cookie=dbf037bdfff9c903 in_seq=0 out_seq=0 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c557fa640 1 -- 172.21.15.132:0/3617912312 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563736c3f470 con 0x563736d53ac0 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c557fa640 10 client.25619 handle_command_reply: tid=0 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 client.25619 shutdown 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 msgr2=0x563736ce10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736ce10a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d10 tx=0x7f6c64005850 comp rx=0 tx=0).stop 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 shutdown_connections 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563736d53ac0 0x563736d55ee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563736cdd5e0 0x563736ce10a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563736cd4810 0x563736ce0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c557fa640 0 client.25619 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 --2- 172.21.15.132:0/3617912312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563736cd3ef0 0x563736ce5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 >> 172.21.15.132:0/3617912312 conn(0x563736cce810 msgr2=0x563736ccfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 shutdown_connections 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 1 -- 172.21.15.132:0/3617912312 wait complete. 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.528+0000 7f6c76d69740 20 client.25619 trim_cache size 0 max 16384 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:14:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:14:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 msgr2=0x7f6c580a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 0x7f6c580a4ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009710 tx=0x7f6c64009ad0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 msgr2=0x7f6c6c0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800ebc0 tx=0x7f6c6800c6a0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 shutdown_connections 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6c580a2a10 0x7f6c580a4ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c6c1056a0 0x7f6c6c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c6c101310 0x7f6c6c0fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 --2- 172.21.15.132:0/1584450007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c6c100970 0x7f6c6c0fe320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 >> 172.21.15.132:0/1584450007 conn(0x7f6c6c0774f0 msgr2=0x7f6c6c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 shutdown_connections 2024-03-18T23:14:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:06.529+0000 7f6c7417e640 1 -- 172.21.15.132:0/1584450007 wait complete. 2024-03-18T23:14:07.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:07 smithi196 ceph-mon[67089]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:14:07.804 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:189/600: open fd = 3 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:190/600: open fd = 3 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:191/600: open fd = 3 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.957 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:192/600: open fd = 3 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:193/600: open fd = 3 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:194/600: open fd = 3 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.958 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:195/600: open fd = 3 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:196/600: open fd = 3 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:197/600: open fd = 3 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.959 INFO:tasks.workunit.client.0.smithi132.stdout:198/600: open fd = 3 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:199/600: open fd = 3 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:200/600: open fd = 3 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:201/600: open fd = 3 2024-03-18T23:14:07.960 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:202/600: open fd = 3 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:203/600: open fd = 3 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:204/600: open fd = 3 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.961 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:205/600: open fd = 3 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:206/600: open fd = 3 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:207/600: open fd = 3 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.962 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:208/600: open fd = 3 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:209/600: open fd = 3 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:210/600: open fd = 3 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:211/600: open fd = 3 2024-03-18T23:14:07.963 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:212/600: open fd = 3 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:213/600: open fd = 3 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:214/600: open fd = 3 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.964 INFO:tasks.workunit.client.0.smithi132.stdout:215/600: open fd = 3 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:216/600: open fd = 3 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:217/600: open fd = 3 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:218/600: open fd = 3 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.965 INFO:tasks.workunit.client.0.smithi132.stdout:219/600: open fd = 3 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:220/600: open fd = 3 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:221/600: open fd = 3 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:222/600: open fd = 3 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.966 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:223/600: open fd = 3 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:224/600: open fd = 3 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:225/600: open fd = 3 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.967 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:226/600: open fd = 3 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:227/600: open fd = 3 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:228/600: open fd = 3 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.968 INFO:tasks.workunit.client.0.smithi132.stdout:229/600: open fd = 3 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:230/600: open fd = 3 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:231/600: open fd = 3 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:232/600: open fd = 3 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.969 INFO:tasks.workunit.client.0.smithi132.stdout:233/600: open fd = 3 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:234/600: open fd = 3 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:235/600: open fd = 3 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:236/600: open fd = 3 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.970 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:237/600: open fd = 3 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:238/600: open fd = 3 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:239/600: open fd = 3 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:240/600: open fd = 3 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.971 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:241/600: open fd = 3 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:242/600: open fd = 3 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:243/600: open fd = 3 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:244/600: open fd = 3 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.972 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:245/600: open fd = 3 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:246/600: open fd = 3 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:247/600: open fd = 3 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:248/600: open fd = 3 2024-03-18T23:14:07.973 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:249/600: open fd = 3 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:250/600: open fd = 3 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.974 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:14:07.975 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:07 smithi144 ceph-mon[66781]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:14:07.975 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:07 smithi132 ceph-mon[32020]: pgmap v149: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-03-18T23:14:09.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:09 smithi196 ceph-mon[67089]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:14:09.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:09 smithi144 ceph-mon[66781]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:14:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:09 smithi132 ceph-mon[32020]: pgmap v150: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-03-18T23:14:11.667 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:11 smithi132 ceph-mon[32020]: pgmap v151: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:11 smithi196 ceph-mon[67089]: pgmap v151: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:11 smithi144 ceph-mon[66781]: pgmap v151: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 31 op/s 2024-03-18T23:14:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:13 smithi196 ceph-mon[67089]: pgmap v152: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:13.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:13 smithi144 ceph-mon[66781]: pgmap v152: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:13 smithi132 ceph-mon[32020]: pgmap v152: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:15 smithi196 ceph-mon[67089]: pgmap v153: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:15 smithi144 ceph-mon[66781]: pgmap v153: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:15.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:15 smithi132 ceph-mon[32020]: pgmap v153: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:17 smithi196 ceph-mon[67089]: pgmap v154: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:17.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:17 smithi144 ceph-mon[66781]: pgmap v154: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:17.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:17 smithi132 ceph-mon[32020]: pgmap v154: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 -- 172.21.15.132:0/1042986047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc1056c0 msgr2=0x7f8ccc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 --2- 172.21.15.132:0/1042986047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc105aa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 -- 172.21.15.132:0/1042986047 shutdown_connections 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 --2- 172.21.15.132:0/1042986047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc101330 0x7f8ccc10d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 --2- 172.21.15.132:0/1042986047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 --2- 172.21.15.132:0/1042986047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc105aa0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 -- 172.21.15.132:0/1042986047 >> 172.21.15.132:0/1042986047 conn(0x7f8ccc077570 msgr2=0x7f8ccc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 -- 172.21.15.132:0/1042986047 shutdown_connections 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 -- 172.21.15.132:0/1042986047 wait complete. 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.625+0000 7f8cd1459640 1 Processor -- start 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 -- start start 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc111d00 con 0x7f8ccc101330 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc111b80 con 0x7f8ccc1056c0 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cd1459640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc111e80 con 0x7f8ccc100970 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58752/0 (socket says 172.21.15.132:58752) 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57406/0 (socket says 172.21.15.132:57406) 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cca7fc640 1 -- 172.21.15.132:0/2862066879 learned_addr learned my addr 172.21.15.132:0/2862066879 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 -- 172.21.15.132:0/2862066879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc1056c0 msgr2=0x7f8ccc19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 -- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 msgr2=0x7f8ccc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8ccaffd640 1 -- 172.21.15.132:0/2862066879 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc19f2c0 con 0x7f8ccc100970 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.626+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc00042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8cb3fff640 1 -- 172.21.15.132:0/2862066879 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0041070 con 0x7f8ccc100970 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc19f550 con 0x7f8ccc100970 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8cb3fff640 1 -- 172.21.15.132:0/2862066879 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc002ccc0 con 0x7f8ccc100970 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8cb3fff640 1 -- 172.21.15.132:0/2862066879 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0003db0 con 0x7f8ccc100970 2024-03-18T23:14:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.627+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a6e40 con 0x7f8ccc100970 2024-03-18T23:14:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.629+0000 7f8cb3fff640 1 -- 172.21.15.132:0/2862066879 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f8cc0044070 con 0x7f8ccc100970 2024-03-18T23:14:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.629+0000 7f8cd4044740 1 Processor -- start 2024-03-18T23:14:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.629+0000 7f8cb3fff640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 0x7f8cb40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.629+0000 7f8cd4044740 1 -- start start 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 0x7f8cb40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee6810 0x55a504eef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cb3fff640 1 -- 172.21.15.132:0/2862066879 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8cc00f06a0 con 0x7f8ccc100970 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504ef19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a504e49be0 con 0x55a504eef5e0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a504df2960 con 0x55a504ee6810 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cd4044740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a504e45c80 con 0x55a504ee5ef0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee6810 0x55a504eef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57422/0 (socket says 172.21.15.132:57422) 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccb7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504ef19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3210425969 learned_addr learned my addr 172.21.15.132:0/3210425969 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3210425969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee6810 msgr2=0x55a504eef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee6810 0x55a504eef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3210425969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 msgr2=0x55a504ef19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504ef19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3210425969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a504de9d20 con 0x55a504ee5ef0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 0x7f8cb40a4ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005fd0 tx=0x7f8cc4013b60 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b9350 tx=0x7f8cc002ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5092017d80f90005 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cb17fa640 1 -- 172.21.15.132:0/3210425969 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf030 con 0x55a504ee5ef0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cb17fa640 1 -- 172.21.15.132:0/3210425969 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.630+0000 7f8cb17fa640 1 -- 172.21.15.132:0/3210425969 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc004b040 con 0x55a504ee5ef0 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- 172.21.15.132:0/3210425969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 msgr2=0x55a504ee62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b9350 tx=0x7f8cc002ce70 comp rx=0 tx=0).stop 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- 172.21.15.132:0/3210425969 shutdown_connections 2024-03-18T23:14:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504ef19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee6810 0x55a504eef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3210425969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504ee62d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- 172.21.15.132:0/3210425969 >> 172.21.15.132:0/3210425969 conn(0x55a504ee0810 msgr2=0x55a504ee0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- 172.21.15.132:0/3210425969 shutdown_connections 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- 172.21.15.132:0/3210425969 wait complete. 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.631+0000 7f8cd4044740 1 -- start start 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 Processor -- start 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504eeefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504eef5e0 0x55a504eec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a504ef1fd0 con 0x55a504ee6810 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a504e05db0 con 0x55a504eef5e0 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a504ef2150 con 0x55a504ee5ef0 2024-03-18T23:14:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cd4044740 10 client.0 _ll_register_callbacks cb 0x55a504e0ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58764/0 (socket says 172.21.15.132:58764) 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cca7fc640 1 -- 172.21.15.132:0/3007141754 learned_addr learned my addr 172.21.15.132:0/3007141754 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504eeefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504eef5e0 0x55a504eec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504eef5e0 msgr2=0x55a504eec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504eef5e0 0x55a504eec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3007141754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 msgr2=0x55a504eec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 -- 172.21.15.132:0/3007141754 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a504e04c60 con 0x55a504ee5ef0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504eeefa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00bc9c0 tx=0x7f8cc0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f8ff79b0b7aa98 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.632+0000 7f8caeff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:14:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf070 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8cc00c5150 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8cc004b070 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 10 client.25630 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a504b5e210 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8cc00bc340 con 0x55a504ee5ef0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 10 client.25630 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 20 client.25630 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504eff010 0x55a504f01400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 4 client.25630 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a504e57120 con 0x55a504eff010 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.633+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504eff010 0x55a504f01400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.634+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504eff010 0x55a504f01400 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e126a847df7c82 server_cookie=b6ab0c8854de4185 in_seq=0 out_seq=0 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.634+0000 7f8caeff5640 10 client.25630 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8caeff5640 1 -- 172.21.15.132:0/3007141754 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a504e57120 con 0x55a504eff010 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8caeff5640 10 client.25630 handle_command_reply: tid=0 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 client.25630 shutdown 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 msgr2=0x55a504eeefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504eeefa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00bc9c0 tx=0x7f8cc0009680 comp rx=0 tx=0).stop 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 shutdown_connections 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504eff010 0x55a504f01400 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504eef5e0 0x55a504eec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504eec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8caeff5640 0 client.25630 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 --2- 172.21.15.132:0/3007141754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee5ef0 0x55a504eeefa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 >> 172.21.15.132:0/3007141754 conn(0x55a504ee0810 msgr2=0x55a504edfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 shutdown_connections 2024-03-18T23:14:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 1 -- 172.21.15.132:0/3007141754 wait complete. 2024-03-18T23:14:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.635+0000 7f8cd4044740 20 client.25630 trim_cache size 0 max 16384 2024-03-18T23:14:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.649+0000 7f8cd4044740 1 Processor -- start 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 -- start start 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504f06af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 0x55a504ef7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b1dd0 con 0x55a504ee6810 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1af6a0 con 0x55a504ee5ef0 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cd4044740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ad4f0 con 0x55a504eef5e0 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504f06af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:46568/0 (socket says 172.21.15.132:46568) 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 -- 172.21.15.132:0/872086043 learned_addr learned my addr 172.21.15.132:0/872086043 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 0x55a504ef7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 -- 172.21.15.132:0/872086043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 msgr2=0x55a504ef7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 0x55a504ef7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 -- 172.21.15.132:0/872086043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 msgr2=0x55a504f06af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504f06af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 -- 172.21.15.132:0/872086043 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a504c2a620 con 0x55a504ee5ef0 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 0x55a504ef7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac07aacda6dbff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cb0ff9640 1 -- 172.21.15.132:0/872086043 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf030 con 0x55a504ee5ef0 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cb0ff9640 1 -- 172.21.15.132:0/872086043 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.650+0000 7f8cb0ff9640 1 -- 172.21.15.132:0/872086043 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00c55e0 con 0x55a504ee5ef0 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- 172.21.15.132:0/872086043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 msgr2=0x55a504ee2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- 172.21.15.132:0/872086043 shutdown_connections 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504eef5e0 0x55a504ef7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504ee6810 0x55a504f06af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 --2- 172.21.15.132:0/872086043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee2370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- 172.21.15.132:0/872086043 >> 172.21.15.132:0/872086043 conn(0x55a504ee0810 msgr2=0x55a504ee1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- 172.21.15.132:0/872086043 shutdown_connections 2024-03-18T23:14:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- 172.21.15.132:0/872086043 wait complete. 2024-03-18T23:14:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 -- start start 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.651+0000 7f8cd4044740 1 Processor -- start 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee6810 0x55a504ee9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1adb60 con 0x55a504eef5e0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1aefd0 con 0x55a504ee5ef0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ae230 con 0x55a504ee6810 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 10 client.0 _ll_register_callbacks cb 0x7f8cc4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccb7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee6810 0x55a504ee9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:46570/0 (socket says 172.21.15.132:46570) 2024-03-18T23:14:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 -- 172.21.15.132:0/410487333 learned_addr learned my addr 172.21.15.132:0/410487333 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccb7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58782/0 (socket says 172.21.15.132:58782) 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 -- 172.21.15.132:0/410487333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee6810 msgr2=0x55a504ee9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee6810 0x55a504ee9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 -- 172.21.15.132:0/410487333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 msgr2=0x55a504f64120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 -- 172.21.15.132:0/410487333 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a504f64760 con 0x55a504ee5ef0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccb7fe640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8ccaffd640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00bcbf0 tx=0x7f8cc002ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8718988ca365b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8caeff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf070 con 0x55a504ee5ef0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8cc00bc340 con 0x55a504ee5ef0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8cc00bcef0 con 0x55a504ee5ef0 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 10 client.25637 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.652+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a504f64760 con 0x55a504ee5ef0 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0005670 con 0x55a504ee5ef0 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8cc00c5150 con 0x55a504ee5ef0 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 10 client.25637 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 20 client.25637 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504f65cc0 0x55a504f680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 4 client.25637 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a504e51470 con 0x55a504f65cc0 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504f65cc0 0x55a504f680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8caeff5640 10 client.25637 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.653+0000 7f8cca7fc640 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504f65cc0 0x55a504f680b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e4b0352849d78d server_cookie=75e1f1b6b2024510 in_seq=0 out_seq=0 2024-03-18T23:14:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.660+0000 7f8caeff5640 1 -- 172.21.15.132:0/410487333 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a504e51470 con 0x55a504f65cc0 2024-03-18T23:14:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.660+0000 7f8caeff5640 10 client.25637 handle_command_reply: tid=0 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.660+0000 7f8cd4044740 1 client.25637 shutdown 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.660+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 msgr2=0x55a504ee9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.660+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00bcbf0 tx=0x7f8cc002ce70 comp rx=0 tx=0).stop 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 shutdown_connections 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a504f65cc0 0x55a504f680b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a504eef5e0 0x55a504f64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a504ee6810 0x55a504ee9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8caeff5640 0 client.25637 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 --2- 172.21.15.132:0/410487333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a504ee5ef0 0x55a504ee9710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 >> 172.21.15.132:0/410487333 conn(0x55a504ee0810 msgr2=0x55a504ee24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 shutdown_connections 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 1 -- 172.21.15.132:0/410487333 wait complete. 2024-03-18T23:14:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd4044740 20 client.25637 trim_cache size 0 max 16384 2024-03-18T23:14:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:14:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:14:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:14:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 msgr2=0x7f8cb40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 0x7f8cb40a4ed0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005fd0 tx=0x7f8cc4013b60 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 msgr2=0x7f8ccc19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc00042e0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 shutdown_connections 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8cb40a2a10 0x7f8cb40a4ed0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ccc1056c0 0x7f8ccc19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ccc101330 0x7f8ccc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 --2- 172.21.15.132:0/2862066879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ccc100970 0x7f8ccc19a2d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 >> 172.21.15.132:0/2862066879 conn(0x7f8ccc077570 msgr2=0x7f8ccc10b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 shutdown_connections 2024-03-18T23:14:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:19.661+0000 7f8cd1459640 1 -- 172.21.15.132:0/2862066879 wait complete. 2024-03-18T23:14:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:19 smithi196 ceph-mon[67089]: pgmap v155: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:19 smithi144 ceph-mon[66781]: pgmap v155: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:19.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:19 smithi132 ceph-mon[32020]: pgmap v155: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:21 smithi196 ceph-mon[67089]: pgmap v156: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-03-18T23:14:21.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:21 smithi144 ceph-mon[66781]: pgmap v156: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-03-18T23:14:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:21 smithi132 ceph-mon[32020]: pgmap v156: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-03-18T23:14:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:23 smithi196 ceph-mon[67089]: pgmap v157: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:23.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:23 smithi144 ceph-mon[66781]: pgmap v157: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:23 smithi132 ceph-mon[32020]: pgmap v157: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:24.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:24 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:24.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:24 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:24.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:24 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:14:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:14:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:25 smithi196 ceph-mon[67089]: pgmap v158: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:14:25.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:25 smithi144 ceph-mon[66781]: pgmap v158: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:14:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:14:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:14:25.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:25 smithi132 ceph-mon[32020]: pgmap v158: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:27.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:27 smithi144 ceph-mon[66781]: pgmap v159: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:27 smithi196 ceph-mon[67089]: pgmap v159: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:27 smithi132 ceph-mon[32020]: pgmap v159: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:29 smithi196 ceph-mon[67089]: pgmap v160: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:29 smithi144 ceph-mon[66781]: pgmap v160: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:29.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:29 smithi132 ceph-mon[32020]: pgmap v160: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:31 smithi196 ceph-mon[67089]: pgmap v161: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:31 smithi144 ceph-mon[66781]: pgmap v161: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:31.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:31 smithi132 ceph-mon[32020]: pgmap v161: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/1993727780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b8100970 msgr2=0x7fe1b8100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/1993727780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b8100970 0x7fe1b8100df0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009890 tx=0x7fe1ac02bb70 comp rx=0 tx=0).stop 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/1993727780 shutdown_connections 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/1993727780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/1993727780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b8100970 0x7fe1b8100df0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/1993727780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b81056c0 0x7fe1b8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/1993727780 >> 172.21.15.132:0/1993727780 conn(0x7fe1b8077570 msgr2=0x7fe1b8077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/1993727780 shutdown_connections 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.757+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/1993727780 wait complete. 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 Processor -- start 2024-03-18T23:14:32.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 -- start start 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b81056c0 0x7fe1b819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8110ba0 con 0x7fe1b8100970 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8110a20 con 0x7fe1b8101330 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1bcf1b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8110d20 con 0x7fe1b81056c0 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52316/0 (socket says 172.21.15.132:52316) 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1b659c640 1 -- 172.21.15.132:0/43025053 learned_addr learned my addr 172.21.15.132:0/43025053 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.758+0000 7fe1b6d9d640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b81056c0 0x7fe1b819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 -- 172.21.15.132:0/43025053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b81056c0 msgr2=0x7fe1b819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b81056c0 0x7fe1b819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 -- 172.21.15.132:0/43025053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 msgr2=0x7fe1b8197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b8197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 -- 172.21.15.132:0/43025053 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b819bff0 con 0x7fe1b8100970 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1b659c640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9c0 tx=0x7fe1a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1a77fe640 1 -- 172.21.15.132:0/43025053 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800cde0 con 0x7fe1b8100970 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b819c2e0 con 0x7fe1b8100970 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1a77fe640 1 -- 172.21.15.132:0/43025053 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a80035f0 con 0x7fe1b8100970 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1a77fe640 1 -- 172.21.15.132:0/43025053 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8010650 con 0x7fe1b8100970 2024-03-18T23:14:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.759+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b81a3b70 con 0x7fe1b8100970 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.761+0000 7fe1a77fe640 1 -- 172.21.15.132:0/43025053 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fe1a80037a0 con 0x7fe1b8100970 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 Processor -- start 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 -- start start 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 0x55987bc8e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 0x55987bc909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55987bbe8be0 con 0x55987bc84ef0 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55987bb91960 con 0x55987bc85810 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55987bbe4c80 con 0x55987bc8e5e0 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 0x55987bc8e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 0x55987bc8e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:38338/0 (socket says 172.21.15.132:38338) 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52322/0 (socket says 172.21.15.132:52322) 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 0x55987bc909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.762+0000 7fe1b659c640 1 -- 172.21.15.132:0/2152420459 learned_addr learned my addr 172.21.15.132:0/2152420459 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1a77fe640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 0x7fe1a00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b6d9d640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 0x7fe1a00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/2152420459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 msgr2=0x55987bc8e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 0x55987bc8e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/2152420459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 msgr2=0x55987bc852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/2152420459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987bb88d20 con 0x55987bc8e5e0 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1a77fe640 1 -- 172.21.15.132:0/43025053 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe1a80cf120 con 0x7fe1b8100970 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 0x55987bc909d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02c180 tx=0x7fe1ac005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d076efd3208be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1a4ff9640 1 -- 172.21.15.132:0/2152420459 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac039040 con 0x55987bc8e5e0 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1b6d9d640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 0x7fe1a00a4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005fd0 tx=0x7fe1b0004ab0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1a4ff9640 1 -- 172.21.15.132:0/2152420459 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac004030 con 0x55987bc8e5e0 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.763+0000 7fe1a4ff9640 1 -- 172.21.15.132:0/2152420459 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac02ca70 con 0x55987bc8e5e0 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/2152420459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 msgr2=0x55987bc909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 0x55987bc909d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02c180 tx=0x7fe1ac005890 comp rx=0 tx=0).stop 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/2152420459 shutdown_connections 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc8e5e0 0x55987bc909d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc85810 0x55987bc8e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/2152420459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/2152420459 >> 172.21.15.132:0/2152420459 conn(0x55987bc7f810 msgr2=0x55987bc7fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/2152420459 shutdown_connections 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.764+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/2152420459 wait complete. 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 -- start start 2024-03-18T23:14:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 Processor -- start 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 0x55987bc8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc8b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55987bc911b0 con 0x55987bc84ef0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55987bba4db0 con 0x55987bc8e5e0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55987bc91330 con 0x55987bc85810 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1bfbe4740 10 client.0 _ll_register_callbacks cb 0x55987bba9e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52338/0 (socket says 172.21.15.132:52338) 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 0x55987bc8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc8b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 -- 172.21.15.132:0/4006824574 learned_addr learned my addr 172.21.15.132:0/4006824574 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 -- 172.21.15.132:0/4006824574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 msgr2=0x55987bc8b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 0x55987bc8b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 -- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 msgr2=0x55987bc8b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc8b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b659c640 1 -- 172.21.15.132:0/4006824574 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987bba3c60 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.765+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 0x55987bc8b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1b659c640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80cc950 tx=0x7fe1a809beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbeaf5480d8b734 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a809f900 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe1a8003b10 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ac004030 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe1a809f120 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 10 client.15804 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55987b8fd210 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac004030 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe1a80a3ac0 con 0x55987bc84ef0 2024-03-18T23:14:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 10 client.15804 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 20 client.15804 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bc9e430 0x55987bca0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 4 client.15804 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987bbf6120 con 0x55987bc9e430 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.766+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bc9e430 0x55987bca0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.767+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bc9e430 0x55987bca0820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcfcb149efe3a6 server_cookie=af9ba5bae8cf5a60 in_seq=0 out_seq=0 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.767+0000 7fe19dffb640 10 client.15804 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.767+0000 7fe19dffb640 1 -- 172.21.15.132:0/4006824574 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55987bbf6120 con 0x55987bc9e430 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.767+0000 7fe19dffb640 10 client.15804 handle_command_reply: tid=0 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.767+0000 7fe1bfbe4740 1 client.15804 shutdown 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 msgr2=0x55987bc8df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80cc950 tx=0x7fe1a809beb0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 shutdown_connections 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bc9e430 0x55987bca0820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc8b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe19dffb640 0 client.15804 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc85810 0x55987bc8b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/4006824574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc84ef0 0x55987bc8df40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 >> 172.21.15.132:0/4006824574 conn(0x55987bc7f810 msgr2=0x55987bc81560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 shutdown_connections 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/4006824574 wait complete. 2024-03-18T23:14:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.768+0000 7fe1bfbe4740 20 client.15804 trim_cache size 0 max 16384 2024-03-18T23:14:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 Processor -- start 2024-03-18T23:14:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 -- start start 2024-03-18T23:14:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bca60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81adae0 con 0x55987bc85810 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81ab3b0 con 0x55987bc8e5e0 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81a9200 con 0x55987bc84ef0 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52340/0 (socket says 172.21.15.132:52340) 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/3633467078 learned_addr learned my addr 172.21.15.132:0/3633467078 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b659c640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b6d9d640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bca60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/3633467078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 msgr2=0x55987bc99f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc99f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/3633467078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 msgr2=0x55987bca60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bca60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/3633467078 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987bc85640 con 0x55987bc85810 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b659c640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc99f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.783+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02c620 tx=0x7fe1ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf1c30e14bef8e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe19ffff640 1 -- 172.21.15.132:0/3633467078 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac039040 con 0x55987bc85810 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe19ffff640 1 -- 172.21.15.132:0/3633467078 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac051380 con 0x55987bc85810 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe19ffff640 1 -- 172.21.15.132:0/3633467078 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0346a0 con 0x55987bc85810 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/3633467078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 msgr2=0x55987bc7fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02c620 tx=0x7fe1ac000bc0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/3633467078 shutdown_connections 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bca60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc7fbf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/3633467078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc99f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/3633467078 >> 172.21.15.132:0/3633467078 conn(0x55987bc7f810 msgr2=0x55987bc7eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/3633467078 shutdown_connections 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/3633467078 wait complete. 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.784+0000 7fe1bfbe4740 1 -- start start 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 Processor -- start 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc92080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81a9870 con 0x55987bc85810 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81aace0 con 0x55987bc8e5e0 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b81a9f40 con 0x55987bc84ef0 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1bfbe4740 10 client.0 _ll_register_callbacks cb 0x7fe1a80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52352/0 (socket says 172.21.15.132:52352) 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/1767714252 learned_addr learned my addr 172.21.15.132:0/1767714252 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b659c640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc92080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b6d9d640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/1767714252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 msgr2=0x55987bc92080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc92080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 msgr2=0x55987bc92b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc92b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 -- 172.21.15.132:0/1767714252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987b9c9620 con 0x55987bc85810 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe1b5d9b640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02bbb0 tx=0x7fe1ac009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e4940d77d87a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe19dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.785+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac039030 con 0x55987bc85810 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe1ac051b10 con 0x55987bc85810 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ac051380 con 0x55987bc85810 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe1ac05f050 con 0x55987bc85810 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55987bd04a40 con 0x55987bc85810 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ac051380 con 0x55987bc85810 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe1ac005040 con 0x55987bc85810 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 20 client.15810 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bd04d80 0x55987bd07170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 4 client.15810 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55987bbf0470 con 0x55987bd04d80 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.786+0000 7fe1b659c640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bd04d80 0x55987bd07170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.787+0000 7fe1b659c640 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bd04d80 0x55987bd07170 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b174adba80a36aa0 server_cookie=ce858cda89b1eafd in_seq=0 out_seq=0 2024-03-18T23:14:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.787+0000 7fe19dffb640 10 client.15810 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe19dffb640 1 -- 172.21.15.132:0/1767714252 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55987bbf0470 con 0x55987bd04d80 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe19dffb640 10 client.15810 handle_command_reply: tid=0 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 client.15810 shutdown 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 msgr2=0x55987bc925c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac02bbb0 tx=0x7fe1ac009500 comp rx=0 tx=0).stop 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 shutdown_connections 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55987bd04d80 0x55987bd07170 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55987bc8e5e0 0x55987bc92b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55987bc85810 0x55987bc925c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe19dffb640 0 client.15810 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 --2- 172.21.15.132:0/1767714252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55987bc84ef0 0x55987bc92080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.794 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 >> 172.21.15.132:0/1767714252 conn(0x55987bc7f810 msgr2=0x55987bc7eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 shutdown_connections 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 1 -- 172.21.15.132:0/1767714252 wait complete. 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.793+0000 7fe1bfbe4740 20 client.15810 trim_cache size 0 max 16384 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:14:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 msgr2=0x7fe1a00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 0x7fe1a00a4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005fd0 tx=0x7fe1b0004ab0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 msgr2=0x7fe1b8197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9c0 tx=0x7fe1a800ee80 comp rx=0 tx=0).stop 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 shutdown_connections 2024-03-18T23:14:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe1a00a2a10 0x7fe1a00a4ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1b81056c0 0x7fe1b819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b8101330 0x7fe1b8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 --2- 172.21.15.132:0/43025053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b8100970 0x7fe1b8197000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 >> 172.21.15.132:0/43025053 conn(0x7fe1b8077570 msgr2=0x7fe1b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 shutdown_connections 2024-03-18T23:14:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:32.794+0000 7fe1bcf1b640 1 -- 172.21.15.132:0/43025053 wait complete. 2024-03-18T23:14:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:33 smithi196 ceph-mon[67089]: pgmap v162: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:33 smithi144 ceph-mon[66781]: pgmap v162: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:33.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:33 smithi132 ceph-mon[32020]: pgmap v162: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:35 smithi196 ceph-mon[67089]: pgmap v163: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:35.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:35 smithi144 ceph-mon[66781]: pgmap v163: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:35.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:35 smithi132 ceph-mon[32020]: pgmap v163: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:14:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:37 smithi196 ceph-mon[67089]: pgmap v164: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:37.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:37 smithi144 ceph-mon[66781]: pgmap v164: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:37.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:37 smithi132 ceph-mon[32020]: pgmap v164: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:39 smithi196 ceph-mon[67089]: pgmap v165: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:39.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:39 smithi144 ceph-mon[66781]: pgmap v165: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:39.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:39 smithi132 ceph-mon[32020]: pgmap v165: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:41.667 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:41 smithi132 ceph-mon[32020]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:41 smithi196 ceph-mon[67089]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:41.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:41 smithi144 ceph-mon[66781]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:43.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:43 smithi196 ceph-mon[67089]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:43 smithi144 ceph-mon[66781]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:43.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:43 smithi132 ceph-mon[32020]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:45 smithi196 ceph-mon[67089]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:45.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:45 smithi144 ceph-mon[66781]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:45.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:45 smithi132 ceph-mon[32020]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 -- 172.21.15.132:0/2434611801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e00069840 msgr2=0x7f8e0010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 --2- 172.21.15.132:0/2434611801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e00069840 0x7f8e0010e6b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009930 tx=0x7f8df802bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 -- 172.21.15.132:0/2434611801 shutdown_connections 2024-03-18T23:14:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 --2- 172.21.15.132:0/2434611801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e00069840 0x7f8e0010e6b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 --2- 172.21.15.132:0/2434611801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00068ea0 0x7f8e00069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 --2- 172.21.15.132:0/2434611801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e001078f0 0x7f8e00107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 -- 172.21.15.132:0/2434611801 >> 172.21.15.132:0/2434611801 conn(0x7f8e00077570 msgr2=0x7f8e00077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 -- 172.21.15.132:0/2434611801 shutdown_connections 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 -- 172.21.15.132:0/2434611801 wait complete. 2024-03-18T23:14:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.891+0000 7f8e05387640 1 Processor -- start 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 -- start start 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e00068ea0 0x7f8e00197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 0x7f8e001984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00111be0 con 0x7f8e001078f0 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00111a60 con 0x7f8e00069840 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8e05387640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e00111d60 con 0x7f8e00068ea0 2024-03-18T23:14:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 0x7f8e001984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37126/0 (socket says 172.21.15.132:37126) 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e00068ea0 0x7f8e00197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dff7fe640 1 -- 172.21.15.132:0/2249745008 learned_addr learned my addr 172.21.15.132:0/2249745008 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 -- 172.21.15.132:0/2249745008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e00068ea0 msgr2=0x7f8e00197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e00068ea0 0x7f8e00197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 -- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 msgr2=0x7f8e0019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dfe7fc640 1 -- 172.21.15.132:0/2249745008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0019cf50 con 0x7f8e00069840 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.892+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 0x7f8e001984a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8df400e9f0 tx=0x7f8df400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8de7fff640 1 -- 172.21.15.132:0/2249745008 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df400ccc0 con 0x7f8e00069840 2024-03-18T23:14:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0019d240 con 0x7f8e00069840 2024-03-18T23:14:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8de7fff640 1 -- 172.21.15.132:0/2249745008 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df4010910 con 0x7f8e00069840 2024-03-18T23:14:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8de7fff640 1 -- 172.21.15.132:0/2249745008 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4018960 con 0x7f8e00069840 2024-03-18T23:14:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.893+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e001a4ad0 con 0x7f8e00069840 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8de7fff640 1 -- 172.21.15.132:0/2249745008 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f8df4010ac0 con 0x7f8e00069840 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 Processor -- start 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 -- start start 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23610ef0 0x556d236112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d2361c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556d23574be0 con 0x556d23611810 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d2351d960 con 0x556d2361a5e0 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.895+0000 7f8e07f72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d23570c80 con 0x556d23610ef0 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37142/0 (socket says 172.21.15.132:37142) 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 -- 172.21.15.132:0/4006193008 learned_addr learned my addr 172.21.15.132:0/4006193008 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23610ef0 0x556d236112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d2361c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8de7fff640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 0x7f8de80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 -- 172.21.15.132:0/4006193008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23610ef0 msgr2=0x556d236112d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23610ef0 0x556d236112d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 -- 172.21.15.132:0/4006193008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 msgr2=0x556d2361c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d2361c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 -- 172.21.15.132:0/4006193008 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d23514d20 con 0x556d23611810 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 0x7f8de80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8de7fff640 1 -- 172.21.15.132:0/2249745008 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8df40cf1f0 con 0x7f8e00069840 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8df00114f0 tx=0x7f8df00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e2ec881cc6ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8de57fa640 1 -- 172.21.15.132:0/4006193008 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df001e070 con 0x556d23611810 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8de57fa640 1 -- 172.21.15.132:0/4006193008 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df000fb00 con 0x556d23611810 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.896+0000 7f8de57fa640 1 -- 172.21.15.132:0/4006193008 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0019b90 con 0x556d23611810 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- 172.21.15.132:0/4006193008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 msgr2=0x556d2361a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8df00114f0 tx=0x7f8df00119b0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 0x7f8de80a4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006010 tx=0x7f8df802e040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- 172.21.15.132:0/4006193008 shutdown_connections 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d2361c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23611810 0x556d2361a0a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 --2- 172.21.15.132:0/4006193008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23610ef0 0x556d236112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- 172.21.15.132:0/4006193008 >> 172.21.15.132:0/4006193008 conn(0x556d2360b810 msgr2=0x556d2360bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- 172.21.15.132:0/4006193008 shutdown_connections 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- 172.21.15.132:0/4006193008 wait complete. 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 -- start start 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.897+0000 7f8e07f72740 1 Processor -- start 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23617340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d2361a5e0 0x556d23617950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x556d2361d210 con 0x556d2361a5e0 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d23530db0 con 0x556d23610ef0 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556d2361d390 con 0x556d23611810 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 10 client.0 _ll_register_callbacks cb 0x556d23535e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d2361a5e0 0x556d23617950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:42130/0 (socket says 172.21.15.132:42130) 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 -- 172.21.15.132:0/539554182 learned_addr learned my addr 172.21.15.132:0/539554182 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23617340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 -- 172.21.15.132:0/539554182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 msgr2=0x556d23617340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23617340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 -- 172.21.15.132:0/539554182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d2361a5e0 msgr2=0x556d23617950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d2361a5e0 0x556d23617950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 -- 172.21.15.132:0/539554182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2352fc60 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8df001c040 tx=0x7f8df000f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed542462a91bac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df001e070 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8df0004c50 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.898+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df000fb00 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8df0004e50 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 10 client.25667 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556d23289210 con 0x556d23610ef0 2024-03-18T23:14:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df000fb00 con 0x556d23610ef0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8df0004020 con 0x556d23610ef0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 10 client.25667 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 20 client.25667 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d2362a3d0 0x556d2362c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 4 client.25667 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d23582120 con 0x556d2362a3d0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d2362a3d0 0x556d2362c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d2362a3d0 0x556d2362c7c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3612f9ed5798a46 server_cookie=29ab96ebd9d51243 in_seq=0 out_seq=0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.899+0000 7f8de2ff5640 10 client.25667 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8de2ff5640 1 -- 172.21.15.132:0/539554182 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556d23582120 con 0x556d2362a3d0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8de2ff5640 10 client.25667 handle_command_reply: tid=0 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 client.25667 shutdown 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 msgr2=0x556d23619fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8df001c040 tx=0x7f8df000f8d0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 shutdown_connections 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d2362a3d0 0x556d2362c7c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d2361a5e0 0x556d23617950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23617340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 --2- 172.21.15.132:0/539554182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d23610ef0 0x556d23619fb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8de2ff5640 0 client.25667 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 >> 172.21.15.132:0/539554182 conn(0x556d2360b810 msgr2=0x556d2360d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 shutdown_connections 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 1 -- 172.21.15.132:0/539554182 wait complete. 2024-03-18T23:14:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.900+0000 7f8e07f72740 20 client.25667 trim_cache size 0 max 16384 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 Processor -- start 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 -- start start 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23631f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d23619a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001afcc0 con 0x556d23610ef0 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001ad590 con 0x556d2361a5e0 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8e07f72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001ab3e0 con 0x556d23611810 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37172/0 (socket says 172.21.15.132:37172) 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 -- 172.21.15.132:0/1960197920 learned_addr learned my addr 172.21.15.132:0/1960197920 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23631f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d23619a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 -- 172.21.15.132:0/1960197920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 msgr2=0x556d23631f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23631f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 -- 172.21.15.132:0/1960197920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 msgr2=0x556d23619a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d23619a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.915+0000 7f8dfeffd640 1 -- 172.21.15.132:0/1960197920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d23355620 con 0x556d23610ef0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f80 tx=0x7f8df0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd12d11c4875dd77 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8de4ff9640 1 -- 172.21.15.132:0/1960197920 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000fae0 con 0x556d23610ef0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8de4ff9640 1 -- 172.21.15.132:0/1960197920 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df0019480 con 0x556d23610ef0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8de4ff9640 1 -- 172.21.15.132:0/1960197920 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003bd0 con 0x556d23610ef0 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 -- 172.21.15.132:0/1960197920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 msgr2=0x556d23625fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8df0000f80 tx=0x7f8df0005cf0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 -- 172.21.15.132:0/1960197920 shutdown_connections 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d23619a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23631f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 --2- 172.21.15.132:0/1960197920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23625fb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 -- 172.21.15.132:0/1960197920 >> 172.21.15.132:0/1960197920 conn(0x556d2360b810 msgr2=0x556d2360aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 -- 172.21.15.132:0/1960197920 shutdown_connections 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.916+0000 7f8e07f72740 1 -- 172.21.15.132:0/1960197920 wait complete. 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 -- start start 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 Processor -- start 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23614380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d236148c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001aba50 con 0x556d23610ef0 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001acec0 con 0x556d2361a5e0 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e001ac120 con 0x556d23611810 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8e07f72740 10 client.0 _ll_register_callbacks cb 0x7f8df001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37182/0 (socket says 172.21.15.132:37182) 2024-03-18T23:14:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 -- 172.21.15.132:0/2192964297 learned_addr learned my addr 172.21.15.132:0/2192964297 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dff7fe640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d236148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23614380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 -- 172.21.15.132:0/2192964297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 msgr2=0x556d23614380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23614380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 -- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 msgr2=0x556d236148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d236148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 -- 172.21.15.132:0/2192964297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2368f530 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8dfeffd640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8df00119f0 tx=0x7f8df0004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530001be7ee6584d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.917+0000 7f8de2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000fae0 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f8df00199c0 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df0019480 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8df0019bc0 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 10 client.15828 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556d23690780 con 0x556d23610ef0 2024-03-18T23:14:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df0019480 con 0x556d23610ef0 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8df0005100 con 0x556d23610ef0 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 20 client.15828 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d23690dd0 0x556d236931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 4 client.15828 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d2357c470 con 0x556d23690dd0 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.918+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d23690dd0 0x556d236931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.919+0000 7f8de2ff5640 10 client.15828 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.919+0000 7f8dfe7fc640 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d23690dd0 0x556d236931f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7178759574529c server_cookie=bec1a7a83cc88111 in_seq=0 out_seq=0 2024-03-18T23:14:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8de2ff5640 1 -- 172.21.15.132:0/2192964297 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d2357c470 con 0x556d23690dd0 2024-03-18T23:14:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8de2ff5640 10 client.15828 handle_command_reply: tid=0 2024-03-18T23:14:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 client.15828 shutdown 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 msgr2=0x556d23613e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8df00119f0 tx=0x7f8df0004540 comp rx=0 tx=0).stop 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 shutdown_connections 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x556d23690dd0 0x556d236931f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d2361a5e0 0x556d236148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556d23611810 0x556d23614380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8de2ff5640 0 client.15828 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 --2- 172.21.15.132:0/2192964297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556d23610ef0 0x556d23613e40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 >> 172.21.15.132:0/2192964297 conn(0x556d2360b810 msgr2=0x556d2360cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 shutdown_connections 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 1 -- 172.21.15.132:0/2192964297 wait complete. 2024-03-18T23:14:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e07f72740 20 client.15828 trim_cache size 0 max 16384 2024-03-18T23:14:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:14:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:14:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:14:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:14:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 msgr2=0x7f8de80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 0x7f8de80a4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006010 tx=0x7f8df802e040 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 msgr2=0x7f8e001984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 0x7f8e001984a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8df400e9f0 tx=0x7f8df400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 shutdown_connections 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.926+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8de80a2a10 0x7f8de80a4ed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e001078f0 0x7f8e0019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e00069840 0x7f8e001984a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 --2- 172.21.15.132:0/2249745008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e00068ea0 0x7f8e00197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 >> 172.21.15.132:0/2249745008 conn(0x7f8e00077570 msgr2=0x7f8e000fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 shutdown_connections 2024-03-18T23:14:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:45.927+0000 7f8e05387640 1 -- 172.21.15.132:0/2249745008 wait complete. 2024-03-18T23:14:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:47 smithi196 ceph-mon[67089]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:14:47.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:47 smithi144 ceph-mon[66781]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:14:47.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:47 smithi132 ceph-mon[32020]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-03-18T23:14:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:49 smithi196 ceph-mon[67089]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:49.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:49 smithi144 ceph-mon[66781]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:49 smithi132 ceph-mon[32020]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:14:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:51 smithi196 ceph-mon[67089]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:51 smithi144 ceph-mon[66781]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:51.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:51 smithi132 ceph-mon[32020]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-03-18T23:14:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:53 smithi196 ceph-mon[67089]: pgmap v172: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:53.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:53 smithi144 ceph-mon[66781]: pgmap v172: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:53.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:53 smithi132 ceph-mon[32020]: pgmap v172: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:55 smithi196 ceph-mon[67089]: pgmap v173: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:55 smithi144 ceph-mon[66781]: pgmap v173: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:55.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:55 smithi132 ceph-mon[32020]: pgmap v173: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:14:57.660 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:57 smithi144 ceph-mon[66781]: pgmap v174: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:57 smithi196 ceph-mon[67089]: pgmap v174: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:57 smithi132 ceph-mon[32020]: pgmap v174: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:14:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 -- 172.21.15.132:0/936159645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c1077e0 msgr2=0x7f3d7c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 --2- 172.21.15.132:0/936159645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c107bc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009930 tx=0x7f3d6c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 -- 172.21.15.132:0/936159645 shutdown_connections 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 --2- 172.21.15.132:0/936159645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c102190 0x7f3d7c10e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 --2- 172.21.15.132:0/936159645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1017d0 0x7f3d7c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 --2- 172.21.15.132:0/936159645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c107bc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 -- 172.21.15.132:0/936159645 >> 172.21.15.132:0/936159645 conn(0x7f3d7c0fd2e0 msgr2=0x7f3d7c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 -- 172.21.15.132:0/936159645 shutdown_connections 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.023+0000 7f3d80a01640 1 -- 172.21.15.132:0/936159645 wait complete. 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 Processor -- start 2024-03-18T23:14:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 -- start start 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c1017d0 0x7f3d7c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c111bb0 con 0x7f3d7c102190 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c111a30 con 0x7f3d7c1077e0 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d80a01640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c111d30 con 0x7f3d7c1017d0 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.024+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51290/0 (socket says 172.21.15.132:51290) 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 -- 172.21.15.132:0/876573443 learned_addr learned my addr 172.21.15.132:0/876573443 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c1017d0 0x7f3d7c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 -- 172.21.15.132:0/876573443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c1017d0 msgr2=0x7f3d7c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c1017d0 0x7f3d7c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 -- 172.21.15.132:0/876573443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 msgr2=0x7f3d7c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 -- 172.21.15.132:0/876573443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c19d020 con 0x7f3d7c102190 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d73fff640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400ea60 tx=0x7f3d6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d737fe640 1 -- 172.21.15.132:0/876573443 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400cde0 con 0x7f3d7c102190 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c19d310 con 0x7f3d7c102190 2024-03-18T23:14:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c1a4ba0 con 0x7f3d7c102190 2024-03-18T23:14:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d737fe640 1 -- 172.21.15.132:0/876573443 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d640035f0 con 0x7f3d7c102190 2024-03-18T23:14:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.025+0000 7f3d737fe640 1 -- 172.21.15.132:0/876573443 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64018690 con 0x7f3d7c102190 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.028+0000 7f3d737fe640 1 -- 172.21.15.132:0/876573443 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f3d64004020 con 0x7f3d7c102190 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.028+0000 7f3d835ea740 1 Processor -- start 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 -- start start 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4e39f0 0x5566bd4e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5566bd439be0 con 0x5566bd4e39f0 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5566bd3e2960 con 0x5566bd4d58b0 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d835ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566bd435c80 con 0x5566bd4d61d0 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d737fe640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 0x7f3d680a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:58700/0 (socket says 172.21.15.132:58700) 2024-03-18T23:14:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 -- 172.21.15.132:0/1993204638 learned_addr learned my addr 172.21.15.132:0/1993204638 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d73fff640 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4e39f0 0x5566bd4e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 0x7f3d680a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d737fe640 1 -- 172.21.15.132:0/876573443 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3d640cefe0 con 0x7f3d7c102190 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 -- 172.21.15.132:0/1993204638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 msgr2=0x5566bd4d5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4d5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 -- 172.21.15.132:0/1993204638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4e39f0 msgr2=0x5566bd4e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4e39f0 0x5566bd4e5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.029+0000 7f3d7a59c640 1 -- 172.21.15.132:0/1993204638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bd3d9d20 con 0x5566bd4d61d0 2024-03-18T23:14:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02cfd0 tx=0x7f3d6c0095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d4c9ede6d7d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 0x7f3d680a4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099e0 tx=0x7f3d74009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d70ff9640 1 -- 172.21.15.132:0/1993204638 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c042070 con 0x5566bd4d61d0 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d70ff9640 1 -- 172.21.15.132:0/1993204638 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d6c004030 con 0x5566bd4d61d0 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d70ff9640 1 -- 172.21.15.132:0/1993204638 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004c00 con 0x5566bd4d61d0 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 -- 172.21.15.132:0/1993204638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 msgr2=0x5566bd4e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02cfd0 tx=0x7f3d6c0095f0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 -- 172.21.15.132:0/1993204638 shutdown_connections 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4e39f0 0x5566bd4e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4d61d0 0x5566bd4e34b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1993204638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.030+0000 7f3d835ea740 1 -- 172.21.15.132:0/1993204638 >> 172.21.15.132:0/1993204638 conn(0x5566bd4d0810 msgr2=0x5566bd4d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- 172.21.15.132:0/1993204638 shutdown_connections 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- 172.21.15.132:0/1993204638 wait complete. 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- start start 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 Processor -- start 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5566bd4e65c0 con 0x5566bd4d58b0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5566bd3f5db0 con 0x5566bd4d61d0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5566bd4e6740 con 0x5566bd4e39f0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d835ea740 10 client.0 _ll_register_callbacks cb 0x5566bd3fae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51306/0 (socket says 172.21.15.132:51306) 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2401208981 learned_addr learned my addr 172.21.15.132:0/2401208981 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.031+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2401208981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 msgr2=0x5566bd4e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 msgr2=0x5566bd4e06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2401208981 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bd3f4c60 con 0x5566bd4d58b0 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02f020 tx=0x7f3d6c004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80eff958122bf6a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c042070 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3d6c02ca90 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d6c004030 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c057050 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d835ea740 10 client.15840 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566bd14e210 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d6c004030 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.032+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3d6c04e070 con 0x5566bd4d58b0 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 10 client.15840 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 20 client.15840 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd4f3690 0x5566bd4f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 4 client.15840 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:14:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566bd447120 con 0x5566bd4f3690 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd4f3690 0x5566bd4f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd4f3690 0x5566bd4f5a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5956bcb3383694 server_cookie=5cd8edc758633801 in_seq=0 out_seq=0 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.033+0000 7f3d61ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2401208981 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566bd447120 con 0x5566bd4f3690 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d61ffb640 10 client.15840 handle_command_reply: tid=0 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 client.15840 shutdown 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 msgr2=0x5566bd4e3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02f020 tx=0x7f3d6c004bf0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 shutdown_connections 2024-03-18T23:14:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd4f3690 0x5566bd4f5a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d61ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2401208981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e3320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 >> 172.21.15.132:0/2401208981 conn(0x5566bd4d0810 msgr2=0x5566bd4d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 shutdown_connections 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 1 -- 172.21.15.132:0/2401208981 wait complete. 2024-03-18T23:14:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.034+0000 7f3d835ea740 20 client.15840 trim_cache size 0 max 16384 2024-03-18T23:14:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 Processor -- start 2024-03-18T23:14:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 -- start start 2024-03-18T23:14:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c010f40 con 0x5566bd4d61d0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00e810 con 0x5566bd4d58b0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d835ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00c660 con 0x5566bd4e39f0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51308/0 (socket says 172.21.15.132:51308) 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 -- 172.21.15.132:0/1572782959 learned_addr learned my addr 172.21.15.132:0/1572782959 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 -- 172.21.15.132:0/1572782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 msgr2=0x5566bd4e2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 -- 172.21.15.132:0/1572782959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 msgr2=0x5566bd4ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4ef360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 -- 172.21.15.132:0/1572782959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bd4d6000 con 0x5566bd4d61d0 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d73fff640 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d64098000 tx=0x7f3d6406e310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa2ab7cb0c84f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.049+0000 7f3d63fff640 1 -- 172.21.15.132:0/1572782959 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6406c040 con 0x5566bd4d61d0 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d63fff640 1 -- 172.21.15.132:0/1572782959 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d6409b630 con 0x5566bd4d61d0 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d63fff640 1 -- 172.21.15.132:0/1572782959 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64097e30 con 0x5566bd4d61d0 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- 172.21.15.132:0/1572782959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 msgr2=0x5566bd4fb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d64098000 tx=0x7f3d6406e310 comp rx=0 tx=0).stop 2024-03-18T23:14:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- 172.21.15.132:0/1572782959 shutdown_connections 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d61d0 0x5566bd4fb140 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 --2- 172.21.15.132:0/1572782959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d58b0 0x5566bd4ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- 172.21.15.132:0/1572782959 >> 172.21.15.132:0/1572782959 conn(0x5566bd4d0810 msgr2=0x5566bd4def80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- 172.21.15.132:0/1572782959 shutdown_connections 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- 172.21.15.132:0/1572782959 wait complete. 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 -- start start 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.050+0000 7f3d835ea740 1 Processor -- start 2024-03-18T23:14:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00ccd0 con 0x5566bd4d58b0 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00e140 con 0x5566bd4d61d0 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00d3a0 con 0x5566bd4e39f0 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 10 client.0 _ll_register_callbacks cb 0x7f3d64009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51310/0 (socket says 172.21.15.132:51310) 2024-03-18T23:14:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2504812464 learned_addr learned my addr 172.21.15.132:0/2504812464 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7ad9d640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2504812464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 msgr2=0x5566bd4e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 msgr2=0x5566bd4e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 -- 172.21.15.132:0/2504812464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bd21a620 con 0x5566bd4d58b0 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d7a59c640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005ec0 tx=0x7f3d6c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab67ecfeba537a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:14:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c042040 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3d6c0038b0 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.051+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d6409b630 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c003ab0 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 10 client.15846 fetch_fsmap learned FSMap version 24 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5566bd559a20 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d6409b630 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3d6c04e050 con 0x5566bd4d58b0 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 10 client.15846 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 20 client.15846 populate_metadata read hostname 'smithi132' 2024-03-18T23:14:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd559ed0 0x5566bd55c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:14:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 4 client.15846 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:14:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566bd441470 con 0x5566bd559ed0 2024-03-18T23:14:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd559ed0 0x5566bd55c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:14:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d61ffb640 10 client.15846 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.052+0000 7f3d73fff640 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd559ed0 0x5566bd55c2f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f7b55b8568ea38 server_cookie=732c994a9d572d6b in_seq=0 out_seq=0 2024-03-18T23:14:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.059+0000 7f3d61ffb640 1 -- 172.21.15.132:0/2504812464 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566bd441470 con 0x5566bd559ed0 2024-03-18T23:14:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.059+0000 7f3d61ffb640 10 client.15846 handle_command_reply: tid=0 2024-03-18T23:14:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.059+0000 7f3d835ea740 1 client.15846 shutdown 2024-03-18T23:14:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.059+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 msgr2=0x5566bd4e74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.059+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005ec0 tx=0x7f3d6c0044f0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 shutdown_connections 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5566bd559ed0 0x5566bd55c2f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5566bd4e39f0 0x5566bd4e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566bd4d61d0 0x5566bd4e79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 --2- 172.21.15.132:0/2504812464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5566bd4d58b0 0x5566bd4e74a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d61ffb640 0 client.15846 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 >> 172.21.15.132:0/2504812464 conn(0x5566bd4d0810 msgr2=0x5566bd4cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 shutdown_connections 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 1 -- 172.21.15.132:0/2504812464 wait complete. 2024-03-18T23:14:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d835ea740 20 client.15846 trim_cache size 0 max 16384 2024-03-18T23:14:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:14:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:14:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:14:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:14:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 msgr2=0x7f3d680a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 0x7f3d680a4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099e0 tx=0x7f3d74009500 comp rx=0 tx=0).stop 2024-03-18T23:14:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 msgr2=0x7f3d7c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:14:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400ea60 tx=0x7f3d6400ef20 comp rx=0 tx=0).stop 2024-03-18T23:14:59.066 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 shutdown_connections 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f3d680a2a10 0x7f3d680a4ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d7c1077e0 0x7f3d7c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c102190 0x7f3d7c198570 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 --2- 172.21.15.132:0/876573443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d7c1017d0 0x7f3d7c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 >> 172.21.15.132:0/876573443 conn(0x7f3d7c0fd2e0 msgr2=0x7f3d7c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 shutdown_connections 2024-03-18T23:14:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:14:59.060+0000 7f3d80a01640 1 -- 172.21.15.132:0/876573443 wait complete. 2024-03-18T23:14:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:14:59 smithi196 ceph-mon[67089]: pgmap v175: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:14:59.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:14:59 smithi132 ceph-mon[32020]: pgmap v175: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:14:59.874 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:14:59 smithi144 ceph-mon[66781]: pgmap v175: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:15:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:01 smithi196 ceph-mon[67089]: pgmap v176: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:01.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:01 smithi144 ceph-mon[66781]: pgmap v176: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:01.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:01 smithi132 ceph-mon[32020]: pgmap v176: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:03 smithi196 ceph-mon[67089]: pgmap v177: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:03.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:03 smithi144 ceph-mon[66781]: pgmap v177: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:03.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:03 smithi132 ceph-mon[32020]: pgmap v177: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:05 smithi196 ceph-mon[67089]: pgmap v178: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:05 smithi144 ceph-mon[66781]: pgmap v178: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:05 smithi132 ceph-mon[32020]: pgmap v178: 97 pgs: 97 active+clean; 3.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:07.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:07 smithi196 ceph-mon[67089]: pgmap v179: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:15:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:07 smithi144 ceph-mon[66781]: pgmap v179: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:15:07.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:07 smithi132 ceph-mon[32020]: pgmap v179: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-03-18T23:15:09.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:09 smithi196 ceph-mon[67089]: pgmap v180: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:09.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:09 smithi144 ceph-mon[66781]: pgmap v180: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:09.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:09 smithi132 ceph-mon[32020]: pgmap v180: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-03-18T23:15:10.132 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.132 INFO:tasks.workunit.client.0.smithi132.stdout:251/600: open fd = 3 2024-03-18T23:15:10.132 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.132 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:252/600: open fd = 3 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:253/600: open fd = 3 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:254/600: open fd = 3 2024-03-18T23:15:10.133 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:255/600: open fd = 3 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:256/600: open fd = 3 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:257/600: open fd = 3 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.134 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:258/600: open fd = 3 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:259/600: open fd = 3 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:260/600: open fd = 3 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.135 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:261/600: open fd = 3 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:262/600: open fd = 3 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:263/600: open fd = 3 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:264/600: open fd = 3 2024-03-18T23:15:10.136 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:265/600: open fd = 3 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:266/600: open fd = 3 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:267/600: open fd = 3 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.137 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:268/600: open fd = 3 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:269/600: open fd = 3 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:270/600: open fd = 3 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.138 INFO:tasks.workunit.client.0.smithi132.stdout:271/600: open fd = 3 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:272/600: open fd = 3 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:273/600: open fd = 3 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:274/600: open fd = 3 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.139 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:275/600: open fd = 3 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:276/600: open fd = 3 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:277/600: open fd = 3 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.140 INFO:tasks.workunit.client.0.smithi132.stdout:278/600: open fd = 3 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:279/600: open fd = 3 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:280/600: open fd = 3 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:281/600: open fd = 3 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.141 INFO:tasks.workunit.client.0.smithi132.stdout:282/600: open fd = 3 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:283/600: open fd = 3 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:284/600: open fd = 3 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:285/600: open fd = 3 2024-03-18T23:15:10.142 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:286/600: open fd = 3 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:287/600: open fd = 3 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:288/600: open fd = 3 2024-03-18T23:15:10.143 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:289/600: open fd = 3 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:290/600: open fd = 3 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.144 INFO:tasks.workunit.client.0.smithi132.stdout:291/600: open fd = 3 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:292/600: open fd = 3 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:293/600: open fd = 3 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:294/600: open fd = 3 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.145 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:295/600: open fd = 3 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:296/600: open fd = 3 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:297/600: open fd = 3 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.146 INFO:tasks.workunit.client.0.smithi132.stdout:298/600: open fd = 3 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:299/600: open fd = 3 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:300/600: open fd = 3 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:301/600: open fd = 3 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.147 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:302/600: open fd = 3 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:303/600: open fd = 3 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:304/600: open fd = 3 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.148 INFO:tasks.workunit.client.0.smithi132.stdout:305/600: open fd = 3 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:306/600: open fd = 3 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:307/600: open fd = 3 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:308/600: open fd = 3 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.149 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:309/600: open fd = 3 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:310/600: open fd = 3 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.150 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:311/600: open fd = 3 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:312/600: open fd = 3 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:10.151 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:15:11.668 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:11 smithi132 ceph-mon[32020]: pgmap v181: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:11 smithi196 ceph-mon[67089]: pgmap v181: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:11.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:11 smithi144 ceph-mon[66781]: pgmap v181: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 31 op/s 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 -- 172.21.15.132:0/2055383950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff006a060 msgr2=0x7faff006a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 --2- 172.21.15.132:0/2055383950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff006a060 0x7faff006a4e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009890 tx=0x7fafd802bb70 comp rx=0 tx=0).stop 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 -- 172.21.15.132:0/2055383950 shutdown_connections 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 --2- 172.21.15.132:0/2055383950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006aa20 0x7faff010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 --2- 172.21.15.132:0/2055383950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff006a060 0x7faff006a4e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 --2- 172.21.15.132:0/2055383950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff01077e0 0x7faff0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 -- 172.21.15.132:0/2055383950 >> 172.21.15.132:0/2055383950 conn(0x7faff0077440 msgr2=0x7faff0077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 -- 172.21.15.132:0/2055383950 shutdown_connections 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.156+0000 7faff4a59640 1 -- 172.21.15.132:0/2055383950 wait complete. 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 Processor -- start 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 -- start start 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 0x7faff0100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 0x7faff0101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111bb0 con 0x7faff006aa20 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111a30 con 0x7faff006a060 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.157+0000 7faff4a59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111d30 con 0x7faff01077e0 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34782/0 (socket says 172.21.15.132:34782) 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 -- 172.21.15.132:0/2616057805 learned_addr learned my addr 172.21.15.132:0/2616057805 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 0x7faff0100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 0x7faff0101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 -- 172.21.15.132:0/2616057805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 msgr2=0x7faff0101550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 0x7faff0101550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 -- 172.21.15.132:0/2616057805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 msgr2=0x7faff0100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 0x7faff0100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 -- 172.21.15.132:0/2616057805 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01a64d0 con 0x7faff006aa20 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafee59c640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 0x7faff0100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 0x7faff0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fafd802c180 tx=0x7fafd8004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe77fe640 1 -- 172.21.15.132:0/2616057805 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8041020 con 0x7faff006aa20 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01a6760 con 0x7faff006aa20 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe77fe640 1 -- 172.21.15.132:0/2616057805 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd802c9f0 con 0x7faff006aa20 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7fafe77fe640 1 -- 172.21.15.132:0/2616057805 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8039040 con 0x7faff006aa20 2024-03-18T23:15:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.158+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a6ca0 con 0x7faff006aa20 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.161+0000 7fafe77fe640 1 -- 172.21.15.132:0/2616057805 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fafd8044020 con 0x7faff006aa20 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.161+0000 7faff7643740 1 Processor -- start 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.161+0000 7faff7643740 1 -- start start 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaea62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeb19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55fedae09be0 con 0x55fedaea5ef0 2024-03-18T23:15:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fedadb2960 con 0x55fedaeaf5e0 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafe77fe640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 0x7fafdc0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7faff7643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fedae05c80 con 0x55fedaea6810 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaea62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:35124/0 (socket says 172.21.15.132:35124) 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 -- 172.21.15.132:0/742992030 learned_addr learned my addr 172.21.15.132:0/742992030 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafe7fff640 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeb19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 0x7fafdc0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafe77fe640 1 -- 172.21.15.132:0/2616057805 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fafd80f0560 con 0x7faff006aa20 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 -- 172.21.15.132:0/742992030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 msgr2=0x55fedaeb19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeb19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 -- 172.21.15.132:0/742992030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 msgr2=0x55fedaea62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaea62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 -- 172.21.15.132:0/742992030 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedada9d20 con 0x55fedaea6810 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.162+0000 7fafee59c640 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fafe0011470 tx=0x7fafe0011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f907a62ae7e629 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 0x7fafdc0a4f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099e0 tx=0x7fafe8009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7fafe4ff9640 1 -- 172.21.15.132:0/742992030 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0005020 con 0x55fedaea6810 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7fafe4ff9640 1 -- 172.21.15.132:0/742992030 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe00047a0 con 0x55fedaea6810 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7fafe4ff9640 1 -- 172.21.15.132:0/742992030 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000fa60 con 0x55fedaea6810 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 -- 172.21.15.132:0/742992030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 msgr2=0x55fedaeaf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fafe0011470 tx=0x7fafe0011930 comp rx=0 tx=0).stop 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 -- 172.21.15.132:0/742992030 shutdown_connections 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeb19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea6810 0x55fedaeaf0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 --2- 172.21.15.132:0/742992030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaea62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.163+0000 7faff7643740 1 -- 172.21.15.132:0/742992030 >> 172.21.15.132:0/742992030 conn(0x55fedaea0810 msgr2=0x55fedaea0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- 172.21.15.132:0/742992030 shutdown_connections 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- 172.21.15.132:0/742992030 wait complete. 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- start start 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 Processor -- start 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeaefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55fedaeb2210 con 0x55fedaea6810 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fedadc5db0 con 0x55fedaeaf5e0 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fedaeb2390 con 0x55fedaea5ef0 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7faff7643740 10 client.0 _ll_register_callbacks cb 0x55fedadcae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34792/0 (socket says 172.21.15.132:34792) 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7fafe7fff640 1 -- 172.21.15.132:0/1418281033 learned_addr learned my addr 172.21.15.132:0/1418281033 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7fafee59c640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeaefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.164+0000 7fafeed9d640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 -- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 msgr2=0x55fedaeac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 -- 172.21.15.132:0/1418281033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 msgr2=0x55fedaeac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 -- 172.21.15.132:0/1418281033 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedadc4c60 con 0x55fedaea5ef0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafe7fff640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafee59c640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeaefc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafe0011440 tx=0x7fafe000fce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98331a7c2555c527 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0028070 con 0x55fedaea5ef0 2024-03-18T23:15:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe00047a0 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fafe0024650 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fafe0034050 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7faff7643740 10 client.25702 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fedab1e210 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.165+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe00047a0 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fafe00226e0 con 0x55fedaea5ef0 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 10 client.25702 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 20 client.25702 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaebf400 0x55fedaec17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 4 client.25702 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fedae17120 con 0x55fedaebf400 2024-03-18T23:15:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7fafe7fff640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaebf400 0x55fedaec17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7fafd5ffb640 10 client.25702 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.166+0000 7fafe7fff640 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaebf400 0x55fedaec17f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0964ac4b3b790bf server_cookie=64df66867a0c2618 in_seq=0 out_seq=0 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7fafd5ffb640 1 -- 172.21.15.132:0/1418281033 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fedae17120 con 0x55fedaebf400 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7fafd5ffb640 10 client.25702 handle_command_reply: tid=0 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 client.25702 shutdown 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 msgr2=0x55fedaeaefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeaefc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafe0011440 tx=0x7fafe000fce0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 shutdown_connections 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaebf400 0x55fedaec17f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaeaf5e0 0x55fedaeac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea6810 0x55fedaeac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7fafd5ffb640 0 client.25702 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 --2- 172.21.15.132:0/1418281033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeaefc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 >> 172.21.15.132:0/1418281033 conn(0x55fedaea0810 msgr2=0x55fedae9fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 shutdown_connections 2024-03-18T23:15:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 1 -- 172.21.15.132:0/1418281033 wait complete. 2024-03-18T23:15:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.167+0000 7faff7643740 20 client.25702 trim_cache size 0 max 16384 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 Processor -- start 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 -- start start 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaeaf5e0 0x55fedaec6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faff01b1ee0 con 0x55fedaea5ef0 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faff01af7b0 con 0x55fedaea6810 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7faff7643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faff01ad600 con 0x55fedaeaf5e0 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34806/0 (socket says 172.21.15.132:34806) 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:37418/0 (socket says 172.21.15.132:37418) 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 -- 172.21.15.132:0/2776095869 learned_addr learned my addr 172.21.15.132:0/2776095869 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaeaf5e0 0x55fedaec6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 -- 172.21.15.132:0/2776095869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaeaf5e0 msgr2=0x55fedaec6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaeaf5e0 0x55fedaec6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 -- 172.21.15.132:0/2776095869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 msgr2=0x55fedae9f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafee59c640 1 -- 172.21.15.132:0/2776095869 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedaeb1d40 con 0x55fedaea5ef0 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.182+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7fafee59c640 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafe0006010 tx=0x7fafe0011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a75781a626bfc9b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7fafd7fff640 1 -- 172.21.15.132:0/2776095869 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0028070 con 0x55fedaea5ef0 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7fafd7fff640 1 -- 172.21.15.132:0/2776095869 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe0022e50 con 0x55fedaea5ef0 2024-03-18T23:15:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7fafd7fff640 1 -- 172.21.15.132:0/2776095869 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0005590 con 0x55fedaea5ef0 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 -- 172.21.15.132:0/2776095869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 msgr2=0x55fedaebafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafe0006010 tx=0x7fafe0011930 comp rx=0 tx=0).stop 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 -- 172.21.15.132:0/2776095869 shutdown_connections 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaeaf5e0 0x55fedaec6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedae9f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 --2- 172.21.15.132:0/2776095869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaea5ef0 0x55fedaebafb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 -- 172.21.15.132:0/2776095869 >> 172.21.15.132:0/2776095869 conn(0x55fedaea0810 msgr2=0x55fedae9f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 -- 172.21.15.132:0/2776095869 shutdown_connections 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.183+0000 7faff7643740 1 -- 172.21.15.132:0/2776095869 wait complete. 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 -- start start 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 Processor -- start 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeb75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedaea89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faff01adc70 con 0x55fedaeaf5e0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faff01af0e0 con 0x55fedaea6810 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faff01ae340 con 0x55fedaea5ef0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7faff7643740 10 client.0 _ll_register_callbacks cb 0x7fafe8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafeed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34818/0 (socket says 172.21.15.132:34818) 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafeed9d640 1 -- 172.21.15.132:0/2637941460 learned_addr learned my addr 172.21.15.132:0/2637941460 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedaea89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafee59c640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeb75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 -- 172.21.15.132:0/2637941460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 msgr2=0x55fedaeb75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeb75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 -- 172.21.15.132:0/2637941460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 msgr2=0x55fedaea8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafe7fff640 1 -- 172.21.15.132:0/2637941460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedabea620 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.184+0000 7fafeed9d640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafe7fff640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedaea89d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fafd8042000 tx=0x7fafd808c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab0655ce2dbff42a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80bc640 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe0022e50 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fafd80c5a50 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fafd8032ce0 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 10 client.25709 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fedaf25070 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe0022e50 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fafd803c030 con 0x55fedaea6810 2024-03-18T23:15:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 10 client.25709 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 20 client.25709 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaf25740 0x55fedaf27b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 4 client.25709 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fedae11470 con 0x55fedaf25740 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.185+0000 7fafee59c640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaf25740 0x55fedaf27b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.186+0000 7fafee59c640 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaf25740 0x55fedaf27b30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80bee5fee716570 server_cookie=d269429daf874258 in_seq=0 out_seq=0 2024-03-18T23:15:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.186+0000 7fafd5ffb640 10 client.25709 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7fafd5ffb640 1 -- 172.21.15.132:0/2637941460 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fedae11470 con 0x55fedaf25740 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7fafd5ffb640 10 client.25709 handle_command_reply: tid=0 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 client.25709 shutdown 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 msgr2=0x55fedaea89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedaea89d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fafd8042000 tx=0x7fafd808c7b0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 shutdown_connections 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55fedaf25740 0x55fedaf27b30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fedaeaf5e0 0x55fedaea8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7fafd5ffb640 0 client.25709 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fedaea6810 0x55fedaea89d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 --2- 172.21.15.132:0/2637941460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fedaea5ef0 0x55fedaeb75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 >> 172.21.15.132:0/2637941460 conn(0x55fedaea0810 msgr2=0x55fedae9fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 shutdown_connections 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 1 -- 172.21.15.132:0/2637941460 wait complete. 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.193+0000 7faff7643740 20 client.25709 trim_cache size 0 max 16384 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:15:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 msgr2=0x7fafdc0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 0x7fafdc0a4f00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fafe80099e0 tx=0x7fafe8009500 comp rx=0 tx=0).stop 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 msgr2=0x7faff0101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fafd802c180 tx=0x7fafd8004550 comp rx=0 tx=0).stop 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 shutdown_connections 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fafdc0a2a40 0x7fafdc0a4f00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faff01077e0 0x7faff0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faff006aa20 0x7faff0101010 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 --2- 172.21.15.132:0/2616057805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faff006a060 0x7faff0100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 >> 172.21.15.132:0/2616057805 conn(0x7faff0077440 msgr2=0x7faff00fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 shutdown_connections 2024-03-18T23:15:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:12.194+0000 7faff4a59640 1 -- 172.21.15.132:0/2616057805 wait complete. 2024-03-18T23:15:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:13 smithi196 ceph-mon[67089]: pgmap v182: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:13 smithi144 ceph-mon[66781]: pgmap v182: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:13.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:13 smithi132 ceph-mon[32020]: pgmap v182: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:15 smithi196 ceph-mon[67089]: pgmap v183: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:15 smithi144 ceph-mon[66781]: pgmap v183: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:15.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:15 smithi132 ceph-mon[32020]: pgmap v183: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-03-18T23:15:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:17 smithi196 ceph-mon[67089]: pgmap v184: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:17.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:17 smithi144 ceph-mon[66781]: pgmap v184: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:17.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:17 smithi132 ceph-mon[32020]: pgmap v184: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:19 smithi196 ceph-mon[67089]: pgmap v185: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:19 smithi144 ceph-mon[66781]: pgmap v185: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:19.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:19 smithi132 ceph-mon[32020]: pgmap v185: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:21 smithi196 ceph-mon[67089]: pgmap v186: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:21.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:21 smithi144 ceph-mon[66781]: pgmap v186: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:21.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:21 smithi132 ceph-mon[32020]: pgmap v186: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:23 smithi196 ceph-mon[67089]: pgmap v187: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:23.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:23 smithi144 ceph-mon[66781]: pgmap v187: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:23.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:23 smithi132 ceph-mon[32020]: pgmap v187: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:25.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:24 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:25.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:24 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:25.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:24 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:15:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 -- 172.21.15.132:0/513543940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed181078f0 msgr2=0x7fed18107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 --2- 172.21.15.132:0/513543940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed181078f0 0x7fed18107cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009930 tx=0x7fed0c02bc30 comp rx=0 tx=0).stop 2024-03-18T23:15:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 -- 172.21.15.132:0/513543940 shutdown_connections 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 --2- 172.21.15.132:0/513543940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18069840 0x7fed1810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 --2- 172.21.15.132:0/513543940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18068ea0 0x7fed18069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 --2- 172.21.15.132:0/513543940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed181078f0 0x7fed18107cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 -- 172.21.15.132:0/513543940 >> 172.21.15.132:0/513543940 conn(0x7fed18077570 msgr2=0x7fed18077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 -- 172.21.15.132:0/513543940 shutdown_connections 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.284+0000 7fed1ed68640 1 -- 172.21.15.132:0/513543940 wait complete. 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 Processor -- start 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 -- start start 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18068ea0 0x7fed18100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed181078f0 0x7fed181014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fed18111be0 con 0x7fed18069840 2024-03-18T23:15:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed18111a60 con 0x7fed18068ea0 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1ed68640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed18111d60 con 0x7fed181078f0 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37718/0 (socket says 172.21.15.132:37718) 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 -- 172.21.15.132:0/1880956047 learned_addr learned my addr 172.21.15.132:0/1880956047 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18068ea0 0x7fed18100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed1d305640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed181078f0 0x7fed181014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 -- 172.21.15.132:0/1880956047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed181078f0 msgr2=0x7fed181014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed181078f0 0x7fed181014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 -- 172.21.15.132:0/1880956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18068ea0 msgr2=0x7fed18100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18068ea0 0x7fed18100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 -- 172.21.15.132:0/1880956047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed181a6520 con 0x7fed18069840 2024-03-18T23:15:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.285+0000 7fed17fff640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d770 tx=0x7fed0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.286+0000 7fed15ffb640 1 -- 172.21.15.132:0/1880956047 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04017070 con 0x7fed18069840 2024-03-18T23:15:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.286+0000 7fed15ffb640 1 -- 172.21.15.132:0/1880956047 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed04005500 con 0x7fed18069840 2024-03-18T23:15:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.286+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed181a6810 con 0x7fed18069840 2024-03-18T23:15:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.286+0000 7fed15ffb640 1 -- 172.21.15.132:0/1880956047 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04004e00 con 0x7fed18069840 2024-03-18T23:15:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.286+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181a6d10 con 0x7fed18069840 2024-03-18T23:15:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.288+0000 7fed15ffb640 1 -- 172.21.15.132:0/1880956047 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fed04004fa0 con 0x7fed18069840 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.288+0000 7fed15ffb640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 0x7fed080a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.288+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 0x7fed080a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.288+0000 7fed15ffb640 1 -- 172.21.15.132:0/1880956047 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fed040ce7d0 con 0x7fed18069840 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 0x7fed080a4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed0c005ab0 tx=0x7fed0c000bc0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 Processor -- start 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 -- start start 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca979810 0x55e7ca9820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca9825e0 0x55e7ca9849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.289+0000 7fed21a2a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca8dcbe0 con 0x55e7ca978ef0 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed21a2a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca885960 con 0x55e7ca979810 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed21a2a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca8d8c80 con 0x55e7ca9825e0 2024-03-18T23:15:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37732/0 (socket says 172.21.15.132:37732) 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 -- 172.21.15.132:0/1199293158 learned_addr learned my addr 172.21.15.132:0/1199293158 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1d305640 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca9825e0 0x55e7ca9849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed17fff640 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca979810 0x55e7ca9820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 -- 172.21.15.132:0/1199293158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca9825e0 msgr2=0x55e7ca9849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca9825e0 0x55e7ca9849d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 -- 172.21.15.132:0/1199293158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca979810 msgr2=0x55e7ca9820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca979810 0x55e7ca9820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 -- 172.21.15.132:0/1199293158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ca87cd20 con 0x55e7ca978ef0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009900 tx=0x7fed0c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bcd2e0f482572 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed02ffd640 1 -- 172.21.15.132:0/1199293158 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c04f070 con 0x55e7ca978ef0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed02ffd640 1 -- 172.21.15.132:0/1199293158 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c004ba0 con 0x55e7ca978ef0 2024-03-18T23:15:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.290+0000 7fed02ffd640 1 -- 172.21.15.132:0/1199293158 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c005570 con 0x55e7ca978ef0 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- 172.21.15.132:0/1199293158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 msgr2=0x55e7ca9792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009900 tx=0x7fed0c03f000 comp rx=0 tx=0).stop 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- 172.21.15.132:0/1199293158 shutdown_connections 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca9825e0 0x55e7ca9849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca979810 0x55e7ca9820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1199293158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca9792d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- 172.21.15.132:0/1199293158 >> 172.21.15.132:0/1199293158 conn(0x55e7ca973810 msgr2=0x55e7ca973bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- 172.21.15.132:0/1199293158 shutdown_connections 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- 172.21.15.132:0/1199293158 wait complete. 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 -- start start 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.291+0000 7fed21a2a740 1 Processor -- start 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca978ef0 0x55e7ca981f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca97f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca9851b0 con 0x55e7ca9825e0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca898db0 con 0x55e7ca978ef0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca985330 con 0x55e7ca979810 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed21a2a740 10 client.0 _ll_register_callbacks cb 0x55e7ca89de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed17fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca97f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37742/0 (socket says 172.21.15.132:37742) 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1cb04640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca978ef0 0x55e7ca981f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 -- 172.21.15.132:0/1696668687 learned_addr learned my addr 172.21.15.132:0/1696668687 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 -- 172.21.15.132:0/1696668687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 msgr2=0x55e7ca97f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca97f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 -- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca978ef0 msgr2=0x55e7ca981f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca978ef0 0x55e7ca981f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 -- 172.21.15.132:0/1696668687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ca897c60 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed17fff640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca97f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fed1d305640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed10011cf0 tx=0x7fed1000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd0032b3be9f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fecf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1000fe60 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.292+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fed10003970 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0c004ba0 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fed10003b70 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 10 client.15876 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7ca5f1210 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c004ba0 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fed1001b600 con 0x55e7ca9825e0 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 20 client.15876 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9924a0 0x55e7ca994890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 4 client.15876 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ca8ea120 con 0x55e7ca9924a0 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.293+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9924a0 0x55e7ca994890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fecf7fff640 10 client.15876 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fed1cb04640 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9924a0 0x55e7ca994890 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c6867f04373937 server_cookie=85250a5471008e0b in_seq=0 out_seq=0 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fecf7fff640 1 -- 172.21.15.132:0/1696668687 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e7ca8ea120 con 0x55e7ca9924a0 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fecf7fff640 10 client.15876 handle_command_reply: tid=0 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fed21a2a740 1 client.15876 shutdown 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 msgr2=0x55e7ca97f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.294+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fed10011cf0 tx=0x7fed1000f790 comp rx=0 tx=0).stop 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 shutdown_connections 2024-03-18T23:15:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9924a0 0x55e7ca994890 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca9825e0 0x55e7ca97f880 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca97f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 --2- 172.21.15.132:0/1696668687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca978ef0 0x55e7ca981f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fecf7fff640 0 client.15876 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 >> 172.21.15.132:0/1696668687 conn(0x55e7ca973810 msgr2=0x55e7ca972ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 shutdown_connections 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 1 -- 172.21.15.132:0/1696668687 wait complete. 2024-03-18T23:15:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.295+0000 7fed21a2a740 20 client.15876 trim_cache size 0 max 16384 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 Processor -- start 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 -- start start 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca99a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca981a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fed181b1f40 con 0x55e7ca978ef0 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed181af810 con 0x55e7ca9825e0 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed21a2a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed181ad660 con 0x55e7ca979810 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed1cb04640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed1cb04640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37744/0 (socket says 172.21.15.132:37744) 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed1cb04640 1 -- 172.21.15.132:0/832468470 learned_addr learned my addr 172.21.15.132:0/832468470 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed1d305640 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca981a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.309+0000 7fed17fff640 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca99a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 -- 172.21.15.132:0/832468470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 msgr2=0x55e7ca99a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca99a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 -- 172.21.15.132:0/832468470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 msgr2=0x55e7ca981a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca981a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 -- 172.21.15.132:0/832468470 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ca6bd620 con 0x55e7ca978ef0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed1cb04640 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004650 tx=0x7fed0c056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec8caf985eab6a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed027fc640 1 -- 172.21.15.132:0/832468470 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c04f050 con 0x55e7ca978ef0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed027fc640 1 -- 172.21.15.132:0/832468470 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c0055d0 con 0x55e7ca978ef0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.310+0000 7fed027fc640 1 -- 172.21.15.132:0/832468470 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c05f030 con 0x55e7ca978ef0 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- 172.21.15.132:0/832468470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 msgr2=0x55e7ca98df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004650 tx=0x7fed0c056040 comp rx=0 tx=0).stop 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- 172.21.15.132:0/832468470 shutdown_connections 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca981a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca979810 0x55e7ca99a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- 172.21.15.132:0/832468470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca978ef0 0x55e7ca98df50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- 172.21.15.132:0/832468470 >> 172.21.15.132:0/832468470 conn(0x55e7ca973810 msgr2=0x55e7ca972ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- 172.21.15.132:0/832468470 shutdown_connections 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- 172.21.15.132:0/832468470 wait complete. 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- start start 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 Processor -- start 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca978ef0 0x55e7ca985730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca9f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fed181adcd0 con 0x55e7ca979810 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed181af140 con 0x55e7ca9825e0 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.311+0000 7fed21a2a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed181ae3a0 con 0x55e7ca978ef0 2024-03-18T23:15:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed21a2a740 10 client.0 _ll_register_callbacks cb 0x7fed10009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:37748/0 (socket says 172.21.15.132:37748) 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 -- 172.21.15.132:0/3510632921 learned_addr learned my addr 172.21.15.132:0/3510632921 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed1cb04640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca978ef0 0x55e7ca985730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed1d305640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca9f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 -- 172.21.15.132:0/3510632921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca978ef0 msgr2=0x55e7ca985730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca978ef0 0x55e7ca985730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 -- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 msgr2=0x55e7ca9f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca9f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 -- 172.21.15.132:0/3510632921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ca9f6d10 con 0x55e7ca979810 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed17fff640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fed04009610 tx=0x7fed04004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3ecd73cb4d8713 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fecf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04020040 con 0x55e7ca979810 2024-03-18T23:15:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fed0409a360 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0c0055d0 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fed040992d0 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed21a2a740 10 client.15882 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7ca9f8060 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.312+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c0055d0 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fed040b3050 con 0x55e7ca979810 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 10 client.15882 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 20 client.15882 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9f87e0 0x55e7ca9fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 4 client.15882 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7ca8e4470 con 0x55e7ca9f87e0 2024-03-18T23:15:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed1cb04640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9f87e0 0x55e7ca9fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fecf7fff640 10 client.15882 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.313+0000 7fed1cb04640 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9f87e0 0x55e7ca9fac00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9bb2c7db7eb7e5 server_cookie=78fb8d3bb1fe4bb0 in_seq=0 out_seq=0 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.320+0000 7fecf7fff640 1 -- 172.21.15.132:0/3510632921 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7ca8e4470 con 0x55e7ca9f87e0 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.320+0000 7fecf7fff640 10 client.15882 handle_command_reply: tid=0 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.320+0000 7fed21a2a740 1 client.15882 shutdown 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 msgr2=0x55e7ca985c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fed04009610 tx=0x7fed04004800 comp rx=0 tx=0).stop 2024-03-18T23:15:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 shutdown_connections 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e7ca9f87e0 0x55e7ca9fac00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7ca9825e0 0x55e7ca9f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fecf7fff640 0 client.15882 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ca979810 0x55e7ca985c70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 --2- 172.21.15.132:0/3510632921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e7ca978ef0 0x55e7ca985730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 >> 172.21.15.132:0/3510632921 conn(0x55e7ca973810 msgr2=0x55e7ca979f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 shutdown_connections 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 1 -- 172.21.15.132:0/3510632921 wait complete. 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed21a2a740 20 client.15882 trim_cache size 0 max 16384 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:15:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:15:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:15:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:15:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 msgr2=0x7fed080a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 0x7fed080a4ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed0c005ab0 tx=0x7fed0c000bc0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 msgr2=0x7fed18100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d770 tx=0x7fed0400dc30 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 shutdown_connections 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fed080a2a10 0x7fed080a4ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed181078f0 0x7fed181014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed18069840 0x7fed18100f80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 --2- 172.21.15.132:0/1880956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed18068ea0 0x7fed18100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.321+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 >> 172.21.15.132:0/1880956047 conn(0x7fed18077570 msgr2=0x7fed180fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.322+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 shutdown_connections 2024-03-18T23:15:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:25.322+0000 7fed1ed68640 1 -- 172.21.15.132:0/1880956047 wait complete. 2024-03-18T23:15:26.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:25 smithi144 ceph-mon[66781]: pgmap v188: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:26.109 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:25 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:15:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:25 smithi132 ceph-mon[32020]: pgmap v188: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:26.115 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:25 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:15:26.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:25 smithi196 ceph-mon[67089]: pgmap v188: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:26.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:15:26.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:15:26.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:25 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:15:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:27 smithi196 ceph-mon[67089]: pgmap v189: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:15:27.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:27 smithi144 ceph-mon[66781]: pgmap v189: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:15:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:27 smithi132 ceph-mon[32020]: pgmap v189: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:15:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:29 smithi196 ceph-mon[67089]: pgmap v190: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:29 smithi144 ceph-mon[66781]: pgmap v190: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:29.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:29 smithi132 ceph-mon[32020]: pgmap v190: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:31 smithi196 ceph-mon[67089]: pgmap v191: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:31 smithi144 ceph-mon[66781]: pgmap v191: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:31.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:31 smithi132 ceph-mon[32020]: pgmap v191: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:33 smithi196 ceph-mon[67089]: pgmap v192: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:33 smithi144 ceph-mon[66781]: pgmap v192: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:33.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:33 smithi132 ceph-mon[32020]: pgmap v192: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:35 smithi196 ceph-mon[67089]: pgmap v193: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:35.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:35 smithi144 ceph-mon[66781]: pgmap v193: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:35.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:35 smithi132 ceph-mon[32020]: pgmap v193: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:37 smithi196 ceph-mon[67089]: pgmap v194: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:37.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:37 smithi144 ceph-mon[66781]: pgmap v194: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:37.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:37 smithi132 ceph-mon[32020]: pgmap v194: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 -- 172.21.15.132:0/3538814405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 msgr2=0x7f6a9410d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/3538814405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a9410d550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009ca0 tx=0x7f6a8c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 -- 172.21.15.132:0/3538814405 shutdown_connections 2024-03-18T23:15:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/3538814405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a9410d550 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/3538814405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/3538814405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a94105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 -- 172.21.15.132:0/3538814405 >> 172.21.15.132:0/3538814405 conn(0x7f6a94077570 msgr2=0x7f6a94077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 -- 172.21.15.132:0/3538814405 shutdown_connections 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 -- 172.21.15.132:0/3538814405 wait complete. 2024-03-18T23:15:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.418+0000 7f6a98b9e640 1 Processor -- start 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 -- start start 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94110be0 con 0x7f6a94101330 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94110a60 con 0x7f6a94100970 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a98b9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94110d60 con 0x7f6a941056c0 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:50968/0 (socket says 172.21.15.132:50968) 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42254/0 (socket says 172.21.15.132:42254) 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 -- 172.21.15.132:0/1312511807 learned_addr learned my addr 172.21.15.132:0/1312511807 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:46374/0 (socket says 172.21.15.132:46374) 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 -- 172.21.15.132:0/1312511807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 msgr2=0x7f6a94198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 -- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 msgr2=0x7f6a94199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a92d9d640 1 -- 172.21.15.132:0/1312511807 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a941a44c0 con 0x7f6a941056c0 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.419+0000 7f6a9259c640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a92d9d640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02c700 tx=0x7f6a8c032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a8bfff640 1 -- 172.21.15.132:0/1312511807 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c042020 con 0x7f6a941056c0 2024-03-18T23:15:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a8bfff640 1 -- 172.21.15.132:0/1312511807 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c034640 con 0x7f6a941056c0 2024-03-18T23:15:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a941a4690 con 0x7f6a941056c0 2024-03-18T23:15:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a8bfff640 1 -- 172.21.15.132:0/1312511807 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c039040 con 0x7f6a941056c0 2024-03-18T23:15:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.420+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a4b00 con 0x7f6a941056c0 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.422+0000 7f6a8bfff640 1 -- 172.21.15.132:0/1312511807 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f6a8c045020 con 0x7f6a941056c0 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.422+0000 7f6a9b789740 1 Processor -- start 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 -- start start 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 0x55a53b9012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b90a5e0 0x55a53b90c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a53b864be0 con 0x55a53b901810 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a53b80d960 con 0x55a53b90a5e0 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9b789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a53b860c80 con 0x55a53b900ef0 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a8bfff640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 0x7f6a800a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42266/0 (socket says 172.21.15.132:42266) 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9259c640 1 -- 172.21.15.132:0/3876279059 learned_addr learned my addr 172.21.15.132:0/3876279059 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 0x55a53b9012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b90a5e0 0x55a53b90c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a92d9d640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 0x7f6a800a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a8bfff640 1 -- 172.21.15.132:0/1312511807 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a8c0f0bc0 con 0x7f6a941056c0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3876279059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b90a5e0 msgr2=0x55a53b90c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b90a5e0 0x55a53b90c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3876279059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 msgr2=0x55a53b90a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3876279059 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53b804d20 con 0x55a53b900ef0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.423+0000 7f6a9259c640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 0x55a53b9012d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6a94069a10 tx=0x7f6a7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853fa9ed985dfb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a92d9d640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 0x7f6a800a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0ee940 tx=0x7f6a8c0dd040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a88d99640 1 -- 172.21.15.132:0/3876279059 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c011050 con 0x55a53b900ef0 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a88d99640 1 -- 172.21.15.132:0/3876279059 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0111f0 con 0x55a53b900ef0 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a88d99640 1 -- 172.21.15.132:0/3876279059 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003830 con 0x55a53b900ef0 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 -- 172.21.15.132:0/3876279059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 msgr2=0x55a53b9012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 0x55a53b9012d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6a94069a10 tx=0x7f6a7c004730 comp rx=0 tx=0).stop 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 -- 172.21.15.132:0/3876279059 shutdown_connections 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b90a5e0 0x55a53b90c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b90a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3876279059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b900ef0 0x55a53b9012d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 -- 172.21.15.132:0/3876279059 >> 172.21.15.132:0/3876279059 conn(0x55a53b8fb810 msgr2=0x55a53b8fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.424+0000 7f6a9b789740 1 -- 172.21.15.132:0/3876279059 shutdown_connections 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 -- 172.21.15.132:0/3876279059 wait complete. 2024-03-18T23:15:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 -- start start 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 Processor -- start 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b907980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a53b90d210 con 0x55a53b901810 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a53b820db0 con 0x55a53b900ef0 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a53b90d390 con 0x55a53b90a5e0 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9b789740 10 client.0 _ll_register_callbacks cb 0x55a53b825e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:46380/0 (socket says 172.21.15.132:46380) 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42280/0 (socket says 172.21.15.132:42280) 2024-03-18T23:15:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a89d9b640 1 -- 172.21.15.132:0/700433334 learned_addr learned my addr 172.21.15.132:0/700433334 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.425+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b907980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 -- 172.21.15.132:0/700433334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 msgr2=0x55a53b907980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b907980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 -- 172.21.15.132:0/700433334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 msgr2=0x55a53b907370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 -- 172.21.15.132:0/700433334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53b81fc60 con 0x55a53b900ef0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a9259c640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c010010 tx=0x7f6a7c00fcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0a7e13ed8f0b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c01f450 con 0x55a53b900ef0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a7c01f5f0 con 0x55a53b900ef0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c0111f0 con 0x55a53b900ef0 2024-03-18T23:15:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c036050 con 0x55a53b900ef0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a9b789740 10 client.25739 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a53b579210 con 0x55a53b900ef0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0111f0 con 0x55a53b900ef0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.426+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6a7c01d020 con 0x55a53b900ef0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 10 client.25739 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 20 client.25739 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b91a320 0x55a53b91c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 4 client.25739 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53b872120 con 0x55a53b91a320 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9259c640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b91a320 0x55a53b91c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a9259c640 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b91a320 0x55a53b91c710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbdf83a5eefd7202 server_cookie=d3387ce676417747 in_seq=0 out_seq=0 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.427+0000 7f6a79ffb640 10 client.25739 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a79ffb640 1 -- 172.21.15.132:0/700433334 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a53b872120 con 0x55a53b91a320 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a79ffb640 10 client.25739 handle_command_reply: tid=0 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 client.25739 shutdown 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 msgr2=0x55a53b909fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c010010 tx=0x7f6a7c00fcf0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 shutdown_connections 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b91a320 0x55a53b91c710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b907980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b901810 0x55a53b907370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a79ffb640 0 client.25739 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 --2- 172.21.15.132:0/700433334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b900ef0 0x55a53b909fe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 >> 172.21.15.132:0/700433334 conn(0x55a53b8fb810 msgr2=0x55a53b8faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 shutdown_connections 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 1 -- 172.21.15.132:0/700433334 wait complete. 2024-03-18T23:15:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.428+0000 7f6a9b789740 20 client.25739 trim_cache size 0 max 16384 2024-03-18T23:15:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 Processor -- start 2024-03-18T23:15:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 -- start start 2024-03-18T23:15:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b921ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b909aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941afc90 con 0x55a53b900ef0 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ad560 con 0x55a53b901810 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9b789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ab3b0 con 0x55a53b90a5e0 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42296/0 (socket says 172.21.15.132:42296) 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a89d9b640 1 -- 172.21.15.132:0/505055323 learned_addr learned my addr 172.21.15.132:0/505055323 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.443+0000 7f6a9259c640 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b921ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b909aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 -- 172.21.15.132:0/505055323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 msgr2=0x55a53b909aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b909aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 -- 172.21.15.132:0/505055323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 msgr2=0x55a53b921ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b921ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 -- 172.21.15.132:0/505055323 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53b645620 con 0x55a53b900ef0 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d396c34b4f97a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a7bfff640 1 -- 172.21.15.132:0/505055323 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00fd60 con 0x55a53b900ef0 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a7bfff640 1 -- 172.21.15.132:0/505055323 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0116a0 con 0x55a53b900ef0 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a7bfff640 1 -- 172.21.15.132:0/505055323 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0052b0 con 0x55a53b900ef0 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a9b789740 1 -- 172.21.15.132:0/505055323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 msgr2=0x55a53b915fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.444+0000 7f6a9b789740 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c014040 comp rx=0 tx=0).stop 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- 172.21.15.132:0/505055323 shutdown_connections 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b909aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b921ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- 172.21.15.132:0/505055323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b915fb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- 172.21.15.132:0/505055323 >> 172.21.15.132:0/505055323 conn(0x55a53b8fb810 msgr2=0x55a53b8faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- 172.21.15.132:0/505055323 shutdown_connections 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- 172.21.15.132:0/505055323 wait complete. 2024-03-18T23:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- start start 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 Processor -- start 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b90e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b90ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941aba20 con 0x55a53b900ef0 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ace90 con 0x55a53b901810 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ac0f0 con 0x55a53b90a5e0 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9b789740 10 client.0 _ll_register_callbacks cb 0x7f6a84009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:42298/0 (socket says 172.21.15.132:42298) 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a9259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b90e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3643172684 learned_addr learned my addr 172.21.15.132:0/3643172684 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b90ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3643172684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 msgr2=0x55a53b90ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b90ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 msgr2=0x55a53b90e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b90e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.445+0000 7f6a89d9b640 1 -- 172.21.15.132:0/3643172684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53b97f7e0 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a89d9b640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0059f0 tx=0x7f6a7c00e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b32a5eb278d9141 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c011070 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a7c011210 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c0116a0 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c011410 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 10 client.15900 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a53b980af0 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c0116a0 con 0x55a53b900ef0 2024-03-18T23:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6a7c01d040 con 0x55a53b900ef0 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 10 client.15900 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 20 client.15900 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b9810f0 0x55a53b9834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 4 client.15900 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a53b86c470 con 0x55a53b9810f0 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.446+0000 7f6a9259c640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b9810f0 0x55a53b9834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.447+0000 7f6a79ffb640 10 client.15900 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.447+0000 7f6a9259c640 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b9810f0 0x55a53b9834e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e816785a968dca server_cookie=bac213484bfa7937 in_seq=0 out_seq=0 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.453+0000 7f6a79ffb640 1 -- 172.21.15.132:0/3643172684 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a53b86c470 con 0x55a53b9810f0 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.453+0000 7f6a79ffb640 10 client.15900 handle_command_reply: tid=0 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.453+0000 7f6a9b789740 1 client.15900 shutdown 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.453+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 msgr2=0x55a53b90e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0059f0 tx=0x7f6a7c00e510 comp rx=0 tx=0).stop 2024-03-18T23:15:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 shutdown_connections 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a53b9810f0 0x55a53b9834e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a53b90a5e0 0x55a53b90ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a79ffb640 0 client.15900 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a53b901810 0x55a53b90e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 --2- 172.21.15.132:0/3643172684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a53b900ef0 0x55a53b90e1c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 >> 172.21.15.132:0/3643172684 conn(0x55a53b8fb810 msgr2=0x55a53b8fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 shutdown_connections 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 1 -- 172.21.15.132:0/3643172684 wait complete. 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a9b789740 20 client.15900 trim_cache size 0 max 16384 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:15:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 msgr2=0x7f6a800a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 0x7f6a800a4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0ee940 tx=0x7f6a8c0dd040 comp rx=0 tx=0).stop 2024-03-18T23:15:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 msgr2=0x7f6a941a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02c700 tx=0x7f6a8c032f20 comp rx=0 tx=0).stop 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 shutdown_connections 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f6a800a2a10 0x7f6a800a4ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a941056c0 0x7f6a941a3f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a94101330 0x7f6a94199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 --2- 172.21.15.132:0/1312511807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a94100970 0x7f6a94198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.454+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 >> 172.21.15.132:0/1312511807 conn(0x7f6a94077570 msgr2=0x7f6a940ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.455+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 shutdown_connections 2024-03-18T23:15:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:38.455+0000 7f6a98b9e640 1 -- 172.21.15.132:0/1312511807 wait complete. 2024-03-18T23:15:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:39 smithi196 ceph-mon[67089]: pgmap v195: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:15:39.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:39 smithi144 ceph-mon[66781]: pgmap v195: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:15:39.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:39 smithi132 ceph-mon[32020]: pgmap v195: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-03-18T23:15:41.667 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:41 smithi132 ceph-mon[32020]: pgmap v196: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:15:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:41 smithi196 ceph-mon[67089]: pgmap v196: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:15:41.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:41 smithi144 ceph-mon[66781]: pgmap v196: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:15:43.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:43 smithi196 ceph-mon[67089]: pgmap v197: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:43 smithi144 ceph-mon[66781]: pgmap v197: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:43.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:43 smithi132 ceph-mon[32020]: pgmap v197: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:45 smithi196 ceph-mon[67089]: pgmap v198: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:45.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:45 smithi144 ceph-mon[66781]: pgmap v198: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:45.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:45 smithi132 ceph-mon[32020]: pgmap v198: 97 pgs: 97 active+clean; 3.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-03-18T23:15:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:47 smithi196 ceph-mon[67089]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:47.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:47 smithi144 ceph-mon[66781]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:47.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:47 smithi132 ceph-mon[32020]: pgmap v199: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:15:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:49 smithi196 ceph-mon[67089]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:49.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:49 smithi144 ceph-mon[66781]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:49 smithi132 ceph-mon[32020]: pgmap v200: 97 pgs: 97 active+clean; 3.8 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 -- 172.21.15.132:0/3220204882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 msgr2=0x7fd97410e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/3220204882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97410e740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009ca0 tx=0x7fd96c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 -- 172.21.15.132:0/3220204882 shutdown_connections 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/3220204882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97410e740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/3220204882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd974100960 0x7fd974106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/3220204882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9740fffb0 0x7fd974100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.549+0000 7fd97a0b8640 1 -- 172.21.15.132:0/3220204882 >> 172.21.15.132:0/3220204882 conn(0x7fd974076ce0 msgr2=0x7fd9740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.550+0000 7fd97a0b8640 1 -- 172.21.15.132:0/3220204882 shutdown_connections 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.550+0000 7fd97a0b8640 1 -- 172.21.15.132:0/3220204882 wait complete. 2024-03-18T23:15:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.550+0000 7fd97a0b8640 1 Processor -- start 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.550+0000 7fd97a0b8640 1 -- start start 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd97419ea90 0x7fd97419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111c20 con 0x7fd9740fffb0 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111aa0 con 0x7fd97419ea90 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd974111da0 con 0x7fd974106bc0 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49480/0 (socket says 172.21.15.132:49480) 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 -- 172.21.15.132:0/2332103704 learned_addr learned my addr 172.21.15.132:0/2332103704 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd973fff640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd97419ea90 0x7fd97419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 -- 172.21.15.132:0/2332103704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 msgr2=0x7fd97419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 -- 172.21.15.132:0/2332103704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd97419ea90 msgr2=0x7fd97419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd97419ea90 0x7fd97419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 -- 172.21.15.132:0/2332103704 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97419f580 con 0x7fd9740fffb0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd970ff9640 1 -- 172.21.15.132:0/2332103704 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400cd80 con 0x7fd9740fffb0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97419f870 con 0x7fd9740fffb0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd970ff9640 1 -- 172.21.15.132:0/2332103704 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd964005500 con 0x7fd9740fffb0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd970ff9640 1 -- 172.21.15.132:0/2332103704 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964010690 con 0x7fd9740fffb0 2024-03-18T23:15:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.551+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9741a7010 con 0x7fd9740fffb0 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd970ff9640 1 -- 172.21.15.132:0/2332103704 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fd9640046c0 con 0x7fd9740fffb0 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 Processor -- start 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 -- start start 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd970ff9640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 0x7fd95c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3301810 0x55e4f330a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f330c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e4f3264be0 con 0x55e4f330a5e0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e4f320d960 con 0x55e4f3300ef0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.554+0000 7fd97cd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4f3260c80 con 0x55e4f3301810 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd973fff640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 0x7fd95c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd970ff9640 1 -- 172.21.15.132:0/2332103704 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd9640cf280 con 0x7fd9740fffb0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:47094/0 (socket says 172.21.15.132:47094) 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 -- 172.21.15.132:0/1372536993 learned_addr learned my addr 172.21.15.132:0/1372536993 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd972ffd640 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3301810 0x55e4f330a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd973fff640 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f330c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 -- 172.21.15.132:0/1372536993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3301810 msgr2=0x55e4f330a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3301810 0x55e4f330a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 -- 172.21.15.132:0/1372536993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 msgr2=0x55e4f330c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f330c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 -- 172.21.15.132:0/1372536993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f3204d20 con 0x55e4f3300ef0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd973fff640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 0x7fd95c0a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd96c004af0 tx=0x7fd96c02cea0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd964098280 tx=0x7fd96409b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8201f7ff167b7d9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd959ffb640 1 -- 172.21.15.132:0/1372536993 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964007b00 con 0x55e4f3300ef0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd959ffb640 1 -- 172.21.15.132:0/1372536993 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96409bfb0 con 0x55e4f3300ef0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.555+0000 7fd959ffb640 1 -- 172.21.15.132:0/1372536993 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96409fa00 con 0x55e4f3300ef0 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- 172.21.15.132:0/1372536993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 msgr2=0x55e4f33012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd964098280 tx=0x7fd96409b3f0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- 172.21.15.132:0/1372536993 shutdown_connections 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f330c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3301810 0x55e4f330a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1372536993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3300ef0 0x55e4f33012d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- 172.21.15.132:0/1372536993 >> 172.21.15.132:0/1372536993 conn(0x55e4f32fb810 msgr2=0x55e4f32fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- 172.21.15.132:0/1372536993 shutdown_connections 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- 172.21.15.132:0/1372536993 wait complete. 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.556+0000 7fd97cd78740 1 -- start start 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 Processor -- start 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f33073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f3307ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e4f330d210 con 0x55e4f330a5e0 2024-03-18T23:15:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e4f3220db0 con 0x55e4f3301810 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e4f330d390 con 0x55e4f3300ef0 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd97cd78740 10 client.0 _ll_register_callbacks cb 0x55e4f3225e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd973fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f3307ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:48864/0 (socket says 172.21.15.132:48864) 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd973fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f3307ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49498/0 (socket says 172.21.15.132:49498) 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 -- 172.21.15.132:0/2190259739 learned_addr learned my addr 172.21.15.132:0/2190259739 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f33073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 -- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 msgr2=0x55e4f33073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f33073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 -- 172.21.15.132:0/2190259739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 msgr2=0x55e4f3307ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f3307ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.565 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 -- 172.21.15.132:0/2190259739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f321fc60 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f33073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd964015040 tx=0x7fd9640056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7142477fa7c0a5c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.557+0000 7fd952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b8070 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd96400f3c0 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96409bfb0 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd9640ccc20 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 10 client.25756 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4f2f79210 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96409bfb0 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd96409bb10 con 0x55e4f3300ef0 2024-03-18T23:15:51.566 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 10 client.25756 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 20 client.25756 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f331a370 0x55e4f331c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 4 client.25756 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4f3272120 con 0x55e4f331a370 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.558+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f331a370 0x55e4f331c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.559+0000 7fd952ffd640 10 client.25756 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.559+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f331a370 0x55e4f331c760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2747a4a1041eb38 server_cookie=3c46cdfa27df7b0a in_seq=0 out_seq=0 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd952ffd640 1 -- 172.21.15.132:0/2190259739 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e4f3272120 con 0x55e4f331a370 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd952ffd640 10 client.25756 handle_command_reply: tid=0 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 client.25756 shutdown 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 msgr2=0x55e4f330a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd964015040 tx=0x7fd9640056b0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 shutdown_connections 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f331a370 0x55e4f331c760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f330a5e0 0x55e4f3307ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f33073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2190259739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f3300ef0 0x55e4f330a010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd952ffd640 0 client.25756 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 >> 172.21.15.132:0/2190259739 conn(0x55e4f32fb810 msgr2=0x55e4f32faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 shutdown_connections 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 1 -- 172.21.15.132:0/2190259739 wait complete. 2024-03-18T23:15:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.560+0000 7fd97cd78740 20 client.25756 trim_cache size 0 max 16384 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.574+0000 7fd97cd78740 1 Processor -- start 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.574+0000 7fd97cd78740 1 -- start start 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f3309ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741b1f40 con 0x55e4f3300ef0 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741af810 con 0x55e4f3301810 2024-03-18T23:15:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd97cd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741ad660 con 0x55e4f32fb810 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd972ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:48870/0 (socket says 172.21.15.132:48870) 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 -- 172.21.15.132:0/2221478245 learned_addr learned my addr 172.21.15.132:0/2221478245 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd972ffd640 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49510/0 (socket says 172.21.15.132:49510) 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd973fff640 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f3309ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 -- 172.21.15.132:0/2221478245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 msgr2=0x55e4f3309ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f3309ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 -- 172.21.15.132:0/2221478245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 msgr2=0x55e4f331f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 -- 172.21.15.132:0/2221478245 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f3301640 con 0x55e4f32fb810 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9737fe640 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd96409b840 tx=0x7fd964004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8521e26ad4687b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9597fa640 1 -- 172.21.15.132:0/2221478245 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b8070 con 0x55e4f32fb810 2024-03-18T23:15:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9597fa640 1 -- 172.21.15.132:0/2221478245 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9640ccdc0 con 0x55e4f32fb810 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.575+0000 7fd9597fa640 1 -- 172.21.15.132:0/2221478245 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96409ad00 con 0x55e4f32fb810 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- 172.21.15.132:0/2221478245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 msgr2=0x55e4f3302c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd96409b840 tx=0x7fd964004b70 comp rx=0 tx=0).stop 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- 172.21.15.132:0/2221478245 shutdown_connections 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f3309ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 --2- 172.21.15.132:0/2221478245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f3302c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- 172.21.15.132:0/2221478245 >> 172.21.15.132:0/2221478245 conn(0x55e4f32f6d20 msgr2=0x55e4f32faab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- 172.21.15.132:0/2221478245 shutdown_connections 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- 172.21.15.132:0/2221478245 wait complete. 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 -- start start 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.576+0000 7fd97cd78740 1 Processor -- start 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f331d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741adcd0 con 0x55e4f3300ef0 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741af140 con 0x55e4f3301810 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9741ae3a0 con 0x55e4f32fb810 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd97cd78740 10 client.0 _ll_register_callbacks cb 0x7fd968009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd972ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:48876/0 (socket says 172.21.15.132:48876) 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 -- 172.21.15.132:0/1708679727 learned_addr learned my addr 172.21.15.132:0/1708679727 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd973fff640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f331d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 -- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 msgr2=0x55e4f331d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f331d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 -- 172.21.15.132:0/1708679727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 msgr2=0x55e4f331cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 -- 172.21.15.132:0/1708679727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f3045620 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd972ffd640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd9737fe640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd964097d10 tx=0x7fd96409b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59050a074f9b741 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.577+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b8050 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd96409ad00 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4f326c470 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd96409bfb0 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 10 client.25762 fetch_fsmap learned FSMap version 24 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4f3045620 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e4f326c470 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd96401b860 con 0x55e4f32fb810 2024-03-18T23:15:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 10 client.25762 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 20 client.25762 populate_metadata read hostname 'smithi132' 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f330a5e0 0x55e4f3382510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 4 client.25762 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4f31ea770 con 0x55e4f330a5e0 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd972ffd640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f330a5e0 0x55e4f3382510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd952ffd640 10 client.25762 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.578+0000 7fd972ffd640 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f330a5e0 0x55e4f3382510 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc832ce23e0757f server_cookie=d473e92339d1f92d in_seq=0 out_seq=0 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd952ffd640 1 -- 172.21.15.132:0/1708679727 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4f31ea770 con 0x55e4f330a5e0 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd952ffd640 10 client.25762 handle_command_reply: tid=0 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 client.25762 shutdown 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 msgr2=0x55e4f331fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd964097d10 tx=0x7fd96409b520 comp rx=0 tx=0).stop 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 shutdown_connections 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e4f330a5e0 0x55e4f3382510 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4f3301810 0x55e4f331d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e4f3300ef0 0x55e4f331cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 --2- 172.21.15.132:0/1708679727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e4f32fb810 0x55e4f331fb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd952ffd640 0 client.25762 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:15:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.585+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 >> 172.21.15.132:0/1708679727 conn(0x55e4f32f6d20 msgr2=0x55e4f33155e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 shutdown_connections 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97cd78740 1 -- 172.21.15.132:0/1708679727 wait complete. 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97cd78740 20 client.25762 trim_cache size 0 max 16384 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:15:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:15:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 msgr2=0x7fd95c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 0x7fd95c0a4ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd96c004af0 tx=0x7fd96c02cea0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 msgr2=0x7fd97419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:15:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400edf0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 shutdown_connections 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd95c0a2a10 0x7fd95c0a4ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd97419ea90 0x7fd97419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd974106bc0 0x7fd97419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 --2- 172.21.15.132:0/2332103704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9740fffb0 0x7fd97419a1d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 >> 172.21.15.132:0/2332103704 conn(0x7fd974076ce0 msgr2=0x7fd974104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 shutdown_connections 2024-03-18T23:15:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:15:51.586+0000 7fd97a0b8640 1 -- 172.21.15.132:0/2332103704 wait complete. 2024-03-18T23:15:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:51 smithi196 ceph-mon[67089]: pgmap v201: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:51 smithi144 ceph-mon[66781]: pgmap v201: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:51.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:51 smithi132 ceph-mon[32020]: pgmap v201: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:15:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:53 smithi196 ceph-mon[67089]: pgmap v202: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:53.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:53 smithi144 ceph-mon[66781]: pgmap v202: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:53.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:53 smithi132 ceph-mon[32020]: pgmap v202: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:55 smithi196 ceph-mon[67089]: pgmap v203: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:55 smithi144 ceph-mon[66781]: pgmap v203: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:55.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:55 smithi132 ceph-mon[32020]: pgmap v203: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:15:57.660 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:57 smithi144 ceph-mon[66781]: pgmap v204: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:57 smithi196 ceph-mon[67089]: pgmap v204: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:57 smithi132 ceph-mon[32020]: pgmap v204: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:15:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:15:59 smithi196 ceph-mon[67089]: pgmap v205: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:59.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:15:59 smithi144 ceph-mon[66781]: pgmap v205: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:15:59.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:15:59 smithi132 ceph-mon[32020]: pgmap v205: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:01 smithi196 ceph-mon[67089]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:16:01.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:01 smithi144 ceph-mon[66781]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:16:01.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:01 smithi132 ceph-mon[32020]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:16:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:03 smithi196 ceph-mon[67089]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:16:03.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:03 smithi144 ceph-mon[66781]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:16:03.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:03 smithi132 ceph-mon[32020]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:16:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.682+0000 7f0e301e6640 1 -- 172.21.15.132:0/1270899835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 msgr2=0x7f0e28107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.682+0000 7f0e301e6640 1 --2- 172.21.15.132:0/1270899835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e28107be0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009930 tx=0x7f0e2402bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- 172.21.15.132:0/1270899835 shutdown_connections 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- 172.21.15.132:0/1270899835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e2810e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- 172.21.15.132:0/1270899835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 0x7f0e28101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- 172.21.15.132:0/1270899835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e28107be0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- 172.21.15.132:0/1270899835 >> 172.21.15.132:0/1270899835 conn(0x7f0e280fd300 msgr2=0x7f0e280ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- 172.21.15.132:0/1270899835 shutdown_connections 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- 172.21.15.132:0/1270899835 wait complete. 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 Processor -- start 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- start start 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 0x7f0e28073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e28074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28111c10 con 0x7f0e28107800 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28111a90 con 0x7f0e281017f0 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.683+0000 7f0e301e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28111d90 con 0x7f0e281021b0 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50808/0 (socket says 172.21.15.132:50808) 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 -- 172.21.15.132:0/77252718 learned_addr learned my addr 172.21.15.132:0/77252718 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2df82640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 0x7f0e28073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2d781640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e28074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 -- 172.21.15.132:0/77252718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 msgr2=0x7f0e28074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e28074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 -- 172.21.15.132:0/77252718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 msgr2=0x7f0e28073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 0x7f0e28073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 -- 172.21.15.132:0/77252718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281a6550 con 0x7f0e28107800 2024-03-18T23:16:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2d781640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e28074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e2e783640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000eb10 tx=0x7f0e2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e1effd640 1 -- 172.21.15.132:0/77252718 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20004b20 con 0x7f0e28107800 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e1effd640 1 -- 172.21.15.132:0/77252718 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e20004cc0 con 0x7f0e28107800 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e281a6840 con 0x7f0e28107800 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e1effd640 1 -- 172.21.15.132:0/77252718 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20010760 con 0x7f0e28107800 2024-03-18T23:16:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.684+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281a6d90 con 0x7f0e28107800 2024-03-18T23:16:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e1effd640 1 -- 172.21.15.132:0/77252718 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f0e20005020 con 0x7f0e28107800 2024-03-18T23:16:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 Processor -- start 2024-03-18T23:16:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 -- start start 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e1effd640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 0x7f0e140a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e66d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55894e5c7be0 con 0x55894e66d5e0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.687+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55894e570960 con 0x55894e663ef0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55894e5c3c80 con 0x55894e664810 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2d781640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 0x7f0e140a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:36802/0 (socket says 172.21.15.132:36802) 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 -- 172.21.15.132:0/1691764877 learned_addr learned my addr 172.21.15.132:0/1691764877 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2d781640 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e66d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e1effd640 1 -- 172.21.15.132:0/77252718 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f0e200cf4b0 con 0x7f0e28107800 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 -- 172.21.15.132:0/1691764877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 msgr2=0x55894e66d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e66d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 -- 172.21.15.132:0/1691764877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 msgr2=0x55894e66f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 -- 172.21.15.132:0/1691764877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55894e567d20 con 0x55894e663ef0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2d781640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 0x7f0e140a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009fd0 tx=0x7f0e18009f40 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e2df82640 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402cfd0 tx=0x7f0e240095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc876cd88bdbb2c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e13fff640 1 -- 172.21.15.132:0/1691764877 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24042070 con 0x55894e663ef0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e13fff640 1 -- 172.21.15.132:0/1691764877 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24004030 con 0x55894e663ef0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.688+0000 7f0e13fff640 1 -- 172.21.15.132:0/1691764877 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004b80 con 0x55894e663ef0 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 -- 172.21.15.132:0/1691764877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 msgr2=0x55894e6642d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402cfd0 tx=0x7f0e240095f0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 -- 172.21.15.132:0/1691764877 shutdown_connections 2024-03-18T23:16:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e66d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/1691764877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6642d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 -- 172.21.15.132:0/1691764877 >> 172.21.15.132:0/1691764877 conn(0x55894e65e810 msgr2=0x55894e65ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 -- 172.21.15.132:0/1691764877 shutdown_connections 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.689+0000 7f0e32dd0740 1 -- 172.21.15.132:0/1691764877 wait complete. 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 -- start start 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 Processor -- start 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e66cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 0x55894e66a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55894e6701b0 con 0x55894e664810 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55894e583db0 con 0x55894e66d5e0 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55894e670330 con 0x55894e663ef0 2024-03-18T23:16:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e32dd0740 10 client.0 _ll_register_callbacks cb 0x55894e588e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e2d781640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e2df82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e66cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e2d781640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50834/0 (socket says 172.21.15.132:50834) 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e2d781640 1 -- 172.21.15.132:0/2975019155 learned_addr learned my addr 172.21.15.132:0/2975019155 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.690+0000 7f0e2e783640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 0x55894e66a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 -- 172.21.15.132:0/2975019155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 msgr2=0x55894e66cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e66cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 -- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 msgr2=0x55894e66a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 0x55894e66a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 -- 172.21.15.132:0/2975019155 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55894e582c60 con 0x55894e664810 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2e783640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 0x55894e66a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e2d781640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e1801e920 tx=0x7f0e1801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f157fed1cadf51a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18024070 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f0e1801ed20 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24004030 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e180208f0 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e32dd0740 10 client.15930 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55894e2dc210 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.691+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24004030 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e18032970 con 0x55894e664810 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 20 client.15930 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e67d510 0x55894e67f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 4 client.15930 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55894e5d5120 con 0x55894e67d510 2024-03-18T23:16:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e2df82640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e67d510 0x55894e67f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e117fa640 10 client.15930 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.692+0000 7f0e2df82640 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e67d510 0x55894e67f900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fc1b1e48c3dc40 server_cookie=736324cc167ec592 in_seq=0 out_seq=0 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e117fa640 1 -- 172.21.15.132:0/2975019155 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55894e5d5120 con 0x55894e67d510 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e117fa640 10 client.15930 handle_command_reply: tid=0 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 client.15930 shutdown 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 msgr2=0x55894e66a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e1801e920 tx=0x7f0e1801d000 comp rx=0 tx=0).stop 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 shutdown_connections 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e67d510 0x55894e67f900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e66d5e0 0x55894e66a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e664810 0x55894e66a360 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2975019155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e66cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e117fa640 0 client.15930 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 >> 172.21.15.132:0/2975019155 conn(0x55894e65e810 msgr2=0x55894e65dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 shutdown_connections 2024-03-18T23:16:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.693+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2975019155 wait complete. 2024-03-18T23:16:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.694+0000 7f0e32dd0740 20 client.15930 trim_cache size 0 max 16384 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 Processor -- start 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 -- start start 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e678f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281b2000 con 0x55894e66d5e0 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281af8d0 con 0x55894e664810 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.708+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281ad740 con 0x55894e663ef0 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2d781640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2df82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e678f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2d781640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:36830/0 (socket says 172.21.15.132:36830) 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2d781640 1 -- 172.21.15.132:0/3996323715 learned_addr learned my addr 172.21.15.132:0/3996323715 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 -- 172.21.15.132:0/3996323715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 msgr2=0x55894e678f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e678f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 -- 172.21.15.132:0/3996323715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 msgr2=0x55894e6851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 -- 172.21.15.132:0/3996323715 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55894e664640 con 0x55894e66d5e0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2d781640 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e2e783640 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66ca90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0e200ccbe0 tx=0x7f0e2000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b901b9e55cbcde5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e137fe640 1 -- 172.21.15.132:0/3996323715 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2009b450 con 0x55894e66d5e0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e137fe640 1 -- 172.21.15.132:0/3996323715 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2009c1f0 con 0x55894e66d5e0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.709+0000 7f0e137fe640 1 -- 172.21.15.132:0/3996323715 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20078f30 con 0x55894e66d5e0 2024-03-18T23:16:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- 172.21.15.132:0/3996323715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 msgr2=0x55894e66ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66ca90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0e200ccbe0 tx=0x7f0e2000cf60 comp rx=0 tx=0).stop 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- 172.21.15.132:0/3996323715 shutdown_connections 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e66ca90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e664810 0x55894e6851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/3996323715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e663ef0 0x55894e678f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- 172.21.15.132:0/3996323715 >> 172.21.15.132:0/3996323715 conn(0x55894e65e810 msgr2=0x55894e65dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- 172.21.15.132:0/3996323715 shutdown_connections 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- 172.21.15.132:0/3996323715 wait complete. 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- start start 2024-03-18T23:16:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 Processor -- start 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6674c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e667a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281add90 con 0x55894e66d5e0 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.710+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281af200 con 0x55894e663ef0 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e32dd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281ae460 con 0x55894e664810 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e32dd0740 10 client.0 _ll_register_callbacks cb 0x7f0e18011560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50860/0 (socket says 172.21.15.132:50860) 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 -- 172.21.15.132:0/2054215430 learned_addr learned my addr 172.21.15.132:0/2054215430 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2df82640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2d781640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e667a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 -- 172.21.15.132:0/2054215430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 msgr2=0x55894e667a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e667a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 -- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 msgr2=0x55894e6674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 -- 172.21.15.132:0/2054215430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55894e3a8620 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e2e783640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e20000f80 tx=0x7f0e2009b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96ff867267e33f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200b4060 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f0e20096170 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e2009c1f0 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e20079110 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e32dd0740 10 client.15936 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55894e6e3cf0 con 0x55894e66d5e0 2024-03-18T23:16:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.711+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2009c1f0 con 0x55894e66d5e0 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e2007a040 con 0x55894e66d5e0 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 10 client.15936 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 20 client.15936 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e6e4310 0x55894e6e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 4 client.15936 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55894e5cf470 con 0x55894e6e4310 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e2df82640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e6e4310 0x55894e6e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e117fa640 10 client.15936 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.712+0000 7f0e2df82640 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e6e4310 0x55894e6e6730 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa61f7e6825a7d7 server_cookie=42926033258b32a0 in_seq=0 out_seq=0 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.719+0000 7f0e117fa640 1 -- 172.21.15.132:0/2054215430 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55894e5cf470 con 0x55894e6e4310 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.719+0000 7f0e117fa640 10 client.15936 handle_command_reply: tid=0 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.719+0000 7f0e32dd0740 1 client.15936 shutdown 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.719+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 msgr2=0x55894e6e27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.719+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0e20000f80 tx=0x7f0e2009b6f0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 shutdown_connections 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55894e6e4310 0x55894e6e6730 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55894e66d5e0 0x55894e6e27d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55894e664810 0x55894e667a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e117fa640 0 client.15936 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 --2- 172.21.15.132:0/2054215430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55894e663ef0 0x55894e6674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 >> 172.21.15.132:0/2054215430 conn(0x55894e65e810 msgr2=0x55894e665020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 shutdown_connections 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 1 -- 172.21.15.132:0/2054215430 wait complete. 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e32dd0740 20 client.15936 trim_cache size 0 max 16384 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:16:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 msgr2=0x7f0e140a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 0x7f0e140a4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009fd0 tx=0x7f0e18009f40 comp rx=0 tx=0).stop 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 msgr2=0x7f0e280745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000eb10 tx=0x7f0e2000c2d0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 shutdown_connections 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e140a2a10 0x7f0e140a4ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e28107800 0x7f0e280745a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e281021b0 0x7f0e28074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 --2- 172.21.15.132:0/77252718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e281017f0 0x7f0e28073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 >> 172.21.15.132:0/77252718 conn(0x7f0e280fd300 msgr2=0x7f0e280fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.720+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 shutdown_connections 2024-03-18T23:16:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:04.721+0000 7f0e301e6640 1 -- 172.21.15.132:0/77252718 wait complete. 2024-03-18T23:16:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:05 smithi196 ceph-mon[67089]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:16:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:05 smithi196 ceph-mon[67089]: pgmap v209: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:05 smithi144 ceph-mon[66781]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:16:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:05 smithi144 ceph-mon[66781]: pgmap v209: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:05.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:05 smithi132 ceph-mon[32020]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T23:16:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:05 smithi132 ceph-mon[32020]: pgmap v209: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:07.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:07 smithi196 ceph-mon[67089]: pgmap v210: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:07 smithi144 ceph-mon[66781]: pgmap v210: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:07.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:07 smithi132 ceph-mon[32020]: pgmap v210: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:16:10.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:09 smithi144 ceph-mon[66781]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:16:10.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:09 smithi144 ceph-mon[66781]: pgmap v212: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:09 smithi132 ceph-mon[32020]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:09 smithi132 ceph-mon[32020]: pgmap v212: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:16:10.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:09 smithi196 ceph-mon[67089]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T23:16:10.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:09 smithi196 ceph-mon[67089]: pgmap v212: 97 pgs: 97 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:16:11.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:11 smithi196 ceph-mon[67089]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:16:11.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:11 smithi144 ceph-mon[66781]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:16:11.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:11 smithi132 ceph-mon[32020]: pgmap v213: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:16:12.479 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.479 INFO:tasks.workunit.client.0.smithi132.stdout:313/600: open fd = 3 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:314/600: open fd = 3 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:315/600: open fd = 3 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:316/600: open fd = 3 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.480 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:317/600: open fd = 3 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:318/600: open fd = 3 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:319/600: open fd = 3 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:320/600: open fd = 3 2024-03-18T23:16:12.481 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:321/600: open fd = 3 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:322/600: open fd = 3 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:323/600: open fd = 3 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.482 INFO:tasks.workunit.client.0.smithi132.stdout:324/600: open fd = 3 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:325/600: open fd = 3 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:326/600: open fd = 3 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:327/600: open fd = 3 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.483 INFO:tasks.workunit.client.0.smithi132.stdout:328/600: open fd = 3 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:329/600: open fd = 3 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:330/600: open fd = 3 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:331/600: open fd = 3 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:332/600: open fd = 3 2024-03-18T23:16:12.484 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:333/600: open fd = 3 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:334/600: open fd = 3 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:335/600: open fd = 3 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:336/600: open fd = 3 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.485 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:337/600: open fd = 3 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:338/600: open fd = 3 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:339/600: open fd = 3 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.486 INFO:tasks.workunit.client.0.smithi132.stdout:340/600: open fd = 3 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:341/600: open fd = 3 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:342/600: open fd = 3 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:343/600: open fd = 3 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.487 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:344/600: open fd = 3 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:345/600: open fd = 3 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:346/600: open fd = 3 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:347/600: open fd = 3 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.488 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:348/600: open fd = 3 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:349/600: open fd = 3 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:350/600: open fd = 3 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.489 INFO:tasks.workunit.client.0.smithi132.stdout:351/600: open fd = 3 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:352/600: open fd = 3 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:353/600: open fd = 3 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:354/600: open fd = 3 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.490 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:355/600: open fd = 3 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:356/600: open fd = 3 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:357/600: open fd = 3 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:358/600: open fd = 3 2024-03-18T23:16:12.491 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:359/600: open fd = 3 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:360/600: open fd = 3 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:361/600: open fd = 3 2024-03-18T23:16:12.492 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:362/600: open fd = 3 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:363/600: open fd = 3 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:364/600: open fd = 3 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.493 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:365/600: open fd = 3 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:366/600: open fd = 3 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:367/600: open fd = 3 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.494 INFO:tasks.workunit.client.0.smithi132.stdout:368/600: open fd = 3 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:369/600: open fd = 3 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:370/600: open fd = 3 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:371/600: open fd = 3 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.495 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:372/600: open fd = 3 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:373/600: open fd = 3 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:374/600: open fd = 3 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:16:12.496 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:16:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:13 smithi196 ceph-mon[67089]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:13 smithi144 ceph-mon[66781]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:13.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:13 smithi132 ceph-mon[32020]: pgmap v214: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:15 smithi196 ceph-mon[67089]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:16:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:15 smithi144 ceph-mon[66781]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:16:15.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:15 smithi132 ceph-mon[32020]: pgmap v215: 97 pgs: 97 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:16:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:17 smithi196 ceph-mon[67089]: pgmap v216: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 -- 172.21.15.132:0/3686934360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8100970 msgr2=0x7f80b8100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/3686934360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8100970 0x7f80b8100dd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009900 tx=0x7f80a402bc30 comp rx=0 tx=0).stop 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 -- 172.21.15.132:0/3686934360 shutdown_connections 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/3686934360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8101310 0x7f80b810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/3686934360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8100970 0x7f80b8100dd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/3686934360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 -- 172.21.15.132:0/3686934360 >> 172.21.15.132:0/3686934360 conn(0x7f80b80774f0 msgr2=0x7f80b8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 -- 172.21.15.132:0/3686934360 shutdown_connections 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.816+0000 7f80bf0b2640 1 -- 172.21.15.132:0/3686934360 wait complete. 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 Processor -- start 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 -- start start 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8100970 0x7f80b8196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8110bb0 con 0x7f80b8101310 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8110a30 con 0x7f80b8100970 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bf0b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8110d30 con 0x7f80b81056a0 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43776/0 (socket says 172.21.15.132:43776) 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 -- 172.21.15.132:0/904657801 learned_addr learned my addr 172.21.15.132:0/904657801 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8100970 0x7f80b8196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bd64f640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 -- 172.21.15.132:0/904657801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 msgr2=0x7f80b819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 -- 172.21.15.132:0/904657801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8100970 msgr2=0x7f80b8196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8100970 0x7f80b8196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80b7fff640 1 -- 172.21.15.132:0/904657801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b819bf50 con 0x7f80b8101310 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.817+0000 7f80bd64f640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80b7fff640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006010 tx=0x7f80a402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80b5ffb640 1 -- 172.21.15.132:0/904657801 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4041070 con 0x7f80b8101310 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80b5ffb640 1 -- 172.21.15.132:0/904657801 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a4004450 con 0x7f80b8101310 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b819c1e0 con 0x7f80b8101310 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80b5ffb640 1 -- 172.21.15.132:0/904657801 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4003e00 con 0x7f80b8101310 2024-03-18T23:16:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.818+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a3ad0 con 0x7f80b8101310 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.820+0000 7f80b5ffb640 1 -- 172.21.15.132:0/904657801 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f80a4044070 con 0x7f80b8101310 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.820+0000 7f80c1d7b740 1 Processor -- start 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 -- start start 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 0x561d4b850d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561d4b7b4be0 con 0x561d4b8512c0 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d4b75d960 con 0x561d4b85c7f0 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4b7b0c80 con 0x561d4b8509a0 2024-03-18T23:16:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80b5ffb640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 0x7f80a80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43778/0 (socket says 172.21.15.132:43778) 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80b7fff640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:58954/0 (socket says 172.21.15.132:58954) 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 -- 172.21.15.132:0/3653085132 learned_addr learned my addr 172.21.15.132:0/3653085132 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bd64f640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 0x561d4b850d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bd64f640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 0x7f80a80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80b5ffb640 1 -- 172.21.15.132:0/904657801 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f80a40f0730 con 0x7f80b8101310 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 -- 172.21.15.132:0/3653085132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 msgr2=0x561d4b850d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 0x561d4b850d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 -- 172.21.15.132:0/3653085132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 msgr2=0x561d4b85ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bce4e640 1 -- 172.21.15.132:0/3653085132 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4b754d20 con 0x561d4b8512c0 2024-03-18T23:16:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80b7fff640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.821+0000 7f80bd64f640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 0x561d4b850d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009d20 tx=0x7f80ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf255a58fe0b0db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80bd64f640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 0x7f80a80a4f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80b00099e0 tx=0x7f80b0009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80a2ffd640 1 -- 172.21.15.132:0/3653085132 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac01e070 con 0x561d4b8512c0 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80a2ffd640 1 -- 172.21.15.132:0/3653085132 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac019b10 con 0x561d4b8512c0 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80a2ffd640 1 -- 172.21.15.132:0/3653085132 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00ebb0 con 0x561d4b8512c0 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3653085132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 msgr2=0x561d4b85c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.822+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009d20 tx=0x7f80ac011810 comp rx=0 tx=0).stop 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3653085132 shutdown_connections 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b85ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8512c0 0x561d4b85c2b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3653085132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8509a0 0x561d4b850d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3653085132 >> 172.21.15.132:0/3653085132 conn(0x561d4b84b810 msgr2=0x561d4b84bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3653085132 shutdown_connections 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3653085132 wait complete. 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- start start 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 Processor -- start 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b859510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b859a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561d4b85f3c0 con 0x561d4b8509a0 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d4b770db0 con 0x561d4b85c7f0 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.823+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561d4b85f540 con 0x561d4b8512c0 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80c1d7b740 10 client.0 _ll_register_callbacks cb 0x561d4b775e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43792/0 (socket says 172.21.15.132:43792) 2024-03-18T23:16:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 -- 172.21.15.132:0/1796725823 learned_addr learned my addr 172.21.15.132:0/1796725823 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bd64f640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b859a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b859510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 -- 172.21.15.132:0/1796725823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 msgr2=0x561d4b859510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b859510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 -- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 msgr2=0x561d4b859a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b859a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 -- 172.21.15.132:0/1796725823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4b76fc60 con 0x561d4b8509a0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f80ac019660 tx=0x7f80ac00e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2b8fb3cd23983 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f8097fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac027020 con 0x561d4b8509a0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f80ac00eaf0 con 0x561d4b8509a0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac019b10 con 0x561d4b8509a0 2024-03-18T23:16:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80ac01e070 con 0x561d4b8509a0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80c1d7b740 10 client.15948 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.824+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561d4b4c9210 con 0x561d4b8509a0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac019b10 con 0x561d4b8509a0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80ac005170 con 0x561d4b8509a0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 10 client.15948 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 20 client.15948 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b86c5c0 0x561d4b86e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 4 client.15948 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d4b7c2120 con 0x561d4b86c5c0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b86c5c0 0x561d4b86e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f8097fff640 10 client.15948 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.825+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b86c5c0 0x561d4b86e9b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61174d4afd7de95 server_cookie=9ee7783b6171a23d in_seq=0 out_seq=0 2024-03-18T23:16:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.826+0000 7f8097fff640 1 -- 172.21.15.132:0/1796725823 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561d4b7c2120 con 0x561d4b86c5c0 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.826+0000 7f8097fff640 10 client.15948 handle_command_reply: tid=0 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.826+0000 7f80c1d7b740 1 client.15948 shutdown 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 msgr2=0x561d4b85c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f80ac019660 tx=0x7f80ac00e490 comp rx=0 tx=0).stop 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 shutdown_connections 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b86c5c0 0x561d4b86e9b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b85c7f0 0x561d4b859a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f8097fff640 0 client.15948 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b859510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1796725823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b8509a0 0x561d4b85c180 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 >> 172.21.15.132:0/1796725823 conn(0x561d4b84b810 msgr2=0x561d4b84d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 shutdown_connections 2024-03-18T23:16:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1796725823 wait complete. 2024-03-18T23:16:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.827+0000 7f80c1d7b740 20 client.15948 trim_cache size 0 max 16384 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.841+0000 7f80c1d7b740 1 Processor -- start 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.841+0000 7f80c1d7b740 1 -- start start 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b8681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ada50 con 0x561d4b85c7f0 2024-03-18T23:16:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ab320 con 0x561d4b8509a0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81a9190 con 0x561d4b8512c0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bd64f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bd64f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43808/0 (socket says 172.21.15.132:43808) 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b8681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:58976/0 (socket says 172.21.15.132:58976) 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bd64f640 1 -- 172.21.15.132:0/3539213718 learned_addr learned my addr 172.21.15.132:0/3539213718 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 -- 172.21.15.132:0/3539213718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 msgr2=0x561d4b8681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b8681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 -- 172.21.15.132:0/3539213718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 msgr2=0x561d4b84a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 -- 172.21.15.132:0/3539213718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d4b859e00 con 0x561d4b8509a0 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bd64f640 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80b7fff640 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b8681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80bce4e640 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80ac011850 tx=0x7f80ac005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738d19111a9eb0af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80a27fc640 1 -- 172.21.15.132:0/3539213718 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac027040 con 0x561d4b8509a0 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80a27fc640 1 -- 172.21.15.132:0/3539213718 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac015070 con 0x561d4b8509a0 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.842+0000 7f80a27fc640 1 -- 172.21.15.132:0/3539213718 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00e880 con 0x561d4b8509a0 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3539213718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 msgr2=0x561d4b864bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80ac011850 tx=0x7f80ac005780 comp rx=0 tx=0).stop 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3539213718 shutdown_connections 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b84a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b8681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/3539213718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b864bb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3539213718 >> 172.21.15.132:0/3539213718 conn(0x561d4b84b810 msgr2=0x561d4b84a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3539213718 shutdown_connections 2024-03-18T23:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- 172.21.15.132:0/3539213718 wait complete. 2024-03-18T23:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 -- start start 2024-03-18T23:16:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.843+0000 7f80c1d7b740 1 Processor -- start 2024-03-18T23:16:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b85b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b85bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81a97e0 con 0x561d4b85c7f0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81aac50 con 0x561d4b8509a0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81a9eb0 con 0x561d4b8512c0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80c1d7b740 10 client.0 _ll_register_callbacks cb 0x7f80ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bd64f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b85bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b85b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bce4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:58980/0 (socket says 172.21.15.132:58980) 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bce4e640 1 -- 172.21.15.132:0/1324538525 learned_addr learned my addr 172.21.15.132:0/1324538525 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 -- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 msgr2=0x561d4b857240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 -- 172.21.15.132:0/1324538525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 msgr2=0x561d4b85bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b85bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 -- 172.21.15.132:0/1324538525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d4b595620 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80bd64f640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b85bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f80b7fff640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b85b490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80a40edf90 tx=0x7f80a4009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d060c34dbfef555 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f8097fff640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.844+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a404b040 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac015070 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f80a402cca0 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80a40c5640 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 10 client.25798 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561d4b8d4710 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac015070 con 0x561d4b8512c0 2024-03-18T23:16:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80a40bd1b0 con 0x561d4b8512c0 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 10 client.25798 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 20 client.25798 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b8d4e40 0x561d4b8d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 4 client.25798 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d4b7bc470 con 0x561d4b8d4e40 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.845+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b8d4e40 0x561d4b8d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.846+0000 7f8097fff640 10 client.25798 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.846+0000 7f80bce4e640 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b8d4e40 0x561d4b8d7260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d416d658ff6701 server_cookie=4419e1176dab2219 in_seq=0 out_seq=0 2024-03-18T23:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f8097fff640 1 -- 172.21.15.132:0/1324538525 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d4b7bc470 con 0x561d4b8d4e40 2024-03-18T23:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f8097fff640 10 client.25798 handle_command_reply: tid=0 2024-03-18T23:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 client.25798 shutdown 2024-03-18T23:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 msgr2=0x561d4b85b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b85b490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80a40edf90 tx=0x7f80a4009560 comp rx=0 tx=0).stop 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 shutdown_connections 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561d4b8d4e40 0x561d4b8d7260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d4b85c7f0 0x561d4b85bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f8097fff640 0 client.25798 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561d4b8512c0 0x561d4b85b490 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 --2- 172.21.15.132:0/1324538525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d4b8509a0 0x561d4b857240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 >> 172.21.15.132:0/1324538525 conn(0x561d4b84b810 msgr2=0x561d4b84aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 shutdown_connections 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 1 -- 172.21.15.132:0/1324538525 wait complete. 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.853+0000 7f80c1d7b740 20 client.25798 trim_cache size 0 max 16384 2024-03-18T23:16:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:16:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 msgr2=0x7f80a80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 0x7f80a80a4f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80b00099e0 tx=0x7f80b0009500 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 msgr2=0x7f80b81974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006010 tx=0x7f80a402cb30 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 shutdown_connections 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80a80a2a90 0x7f80a80a4f50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80b81056a0 0x7f80b819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b8101310 0x7f80b81974f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 --2- 172.21.15.132:0/904657801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b8100970 0x7f80b8196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 >> 172.21.15.132:0/904657801 conn(0x7f80b80774f0 msgr2=0x7f80b80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 shutdown_connections 2024-03-18T23:16:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:17.854+0000 7f80bf0b2640 1 -- 172.21.15.132:0/904657801 wait complete. 2024-03-18T23:16:17.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:17 smithi144 ceph-mon[66781]: pgmap v216: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:16:17.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:17 smithi132 ceph-mon[32020]: pgmap v216: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-03-18T23:16:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:19 smithi196 ceph-mon[67089]: pgmap v217: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-18T23:16:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:19 smithi144 ceph-mon[66781]: pgmap v217: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-18T23:16:19.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:19 smithi132 ceph-mon[32020]: pgmap v217: 97 pgs: 97 active+clean; 4.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-18T23:16:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:21 smithi196 ceph-mon[67089]: pgmap v218: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:21.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:21 smithi144 ceph-mon[66781]: pgmap v218: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:21.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:21 smithi132 ceph-mon[32020]: pgmap v218: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:23 smithi196 ceph-mon[67089]: pgmap v219: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:23.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:23 smithi144 ceph-mon[66781]: pgmap v219: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:23.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:23 smithi132 ceph-mon[32020]: pgmap v219: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:25 smithi196 ceph-mon[67089]: pgmap v220: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:25 smithi144 ceph-mon[66781]: pgmap v220: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:25.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:25 smithi132 ceph-mon[32020]: pgmap v220: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:26.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:26 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:26.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:26 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:26.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:26 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:26.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:26 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:16:26.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:26 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:26.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:26 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:26.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:26 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:26.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:26 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:16:26.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:26 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:16:26.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:26 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:16:26.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:26 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:16:26.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:26 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:16:27.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:27 smithi196 ceph-mon[67089]: pgmap v221: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:27.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:27 smithi144 ceph-mon[66781]: pgmap v221: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:27.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:27 smithi132 ceph-mon[32020]: pgmap v221: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:29 smithi196 ceph-mon[67089]: pgmap v222: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:29 smithi144 ceph-mon[66781]: pgmap v222: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:29.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:29 smithi132 ceph-mon[32020]: pgmap v222: 97 pgs: 97 active+clean; 4.1 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 -- 172.21.15.132:0/2684488703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f299406acc0 msgr2=0x7f2994104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 --2- 172.21.15.132:0/2684488703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f299406acc0 0x7f2994104300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f298c009ca0 tx=0x7f298c02c180 comp rx=0 tx=0).stop 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 -- 172.21.15.132:0/2684488703 shutdown_connections 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 --2- 172.21.15.132:0/2684488703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f299406acc0 0x7f2994104300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 --2- 172.21.15.132:0/2684488703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299406a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 --2- 172.21.15.132:0/2684488703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299406a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 -- 172.21.15.132:0/2684488703 >> 172.21.15.132:0/2684488703 conn(0x7f2994077560 msgr2=0x7f2994077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 -- 172.21.15.132:0/2684488703 shutdown_connections 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.949+0000 7f299b77e640 1 -- 172.21.15.132:0/2684488703 wait complete. 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 Processor -- start 2024-03-18T23:16:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 -- start start 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 0x7f29941a3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2994118240 con 0x7f2994106b80 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29941180c0 con 0x7f29941a2e80 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299b77e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29941183c0 con 0x7f29941067a0 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f2998d19640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f2998d19640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58642/0 (socket says 172.21.15.132:58642) 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f2998d19640 1 -- 172.21.15.132:0/2793978493 learned_addr learned my addr 172.21.15.132:0/2793978493 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f299951a640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.950+0000 7f2999d1b640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 0x7f29941a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 -- 172.21.15.132:0/2793978493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 msgr2=0x7f299419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 -- 172.21.15.132:0/2793978493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 msgr2=0x7f29941a3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 0x7f29941a3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 -- 172.21.15.132:0/2793978493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29941a3970 con 0x7f2994106b80 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2999d1b640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 0x7f29941a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f299951a640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f2998d19640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f298400e9c0 tx=0x7f298400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f298a7fc640 1 -- 172.21.15.132:0/2793978493 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298400ccc0 con 0x7f2994106b80 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29941a3c60 con 0x7f2994106b80 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f298a7fc640 1 -- 172.21.15.132:0/2793978493 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2984005500 con 0x7f2994106b80 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f298a7fc640 1 -- 172.21.15.132:0/2793978493 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984010430 con 0x7f2994106b80 2024-03-18T23:16:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.951+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941ab450 con 0x7f2994106b80 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.953+0000 7f298a7fc640 1 -- 172.21.15.132:0/2793978493 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f29840046c0 con 0x7f2994106b80 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 Processor -- start 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 -- start start 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f298a7fc640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 0x7f29800a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x562afe654be0 con 0x562afe6f0ef0 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562afe5fd960 con 0x562afe6f1810 2024-03-18T23:16:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299e36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562afe650c80 con 0x562afe6fa5e0 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f2999d1b640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 0x7f29800a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299951a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299951a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58652/0 (socket says 172.21.15.132:58652) 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f299951a640 1 -- 172.21.15.132:0/1948092163 learned_addr learned my addr 172.21.15.132:0/1948092163 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f2998d19640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f298a7fc640 1 -- 172.21.15.132:0/2793978493 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f29840cf120 con 0x7f2994106b80 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.954+0000 7f2999d1b640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 -- 172.21.15.132:0/1948092163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 msgr2=0x562afe6fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 -- 172.21.15.132:0/1948092163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 msgr2=0x562afe6f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 -- 172.21.15.132:0/1948092163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afe5f4d20 con 0x562afe6f1810 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f299951a640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2999d1b640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 0x7f29800a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f298c004af0 tx=0x7f298c0323c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f2998d19640 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fa0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29840cc870 tx=0x7f2984097ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd63d052d29df7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f297f7fe640 1 -- 172.21.15.132:0/1948092163 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298409b0e0 con 0x562afe6f1810 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f297f7fe640 1 -- 172.21.15.132:0/1948092163 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298409be80 con 0x562afe6f1810 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.955+0000 7f297f7fe640 1 -- 172.21.15.132:0/1948092163 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298407f460 con 0x562afe6f1810 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- 172.21.15.132:0/1948092163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 msgr2=0x562afe6fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fa0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29840cc870 tx=0x7f2984097ae0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- 172.21.15.132:0/1948092163 shutdown_connections 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fa0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 --2- 172.21.15.132:0/1948092163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe6f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- 172.21.15.132:0/1948092163 >> 172.21.15.132:0/1948092163 conn(0x562afe6eb810 msgr2=0x562afe6ebbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- 172.21.15.132:0/1948092163 shutdown_connections 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- 172.21.15.132:0/1948092163 wait complete. 2024-03-18T23:16:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 -- start start 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.956+0000 7f299e36c740 1 Processor -- start 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 0x562afe6f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x562afe6fd210 con 0x562afe6fa5e0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562afe610db0 con 0x562afe6f0ef0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562afe6fd390 con 0x562afe6f1810 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 10 client.0 _ll_register_callbacks cb 0x562afe615e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f2999d1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f2998d19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 0x562afe6f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:35734/0 (socket says 172.21.15.132:35734) 2024-03-18T23:16:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 -- 172.21.15.132:0/1527306646 learned_addr learned my addr 172.21.15.132:0/1527306646 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 -- 172.21.15.132:0/1527306646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 msgr2=0x562afe6f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 0x562afe6f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 -- 172.21.15.132:0/1527306646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 msgr2=0x562afe6f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 -- 172.21.15.132:0/1527306646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afe60fc60 con 0x562afe6f0ef0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f2998d19640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 0x562afe6f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299951a640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2990012d10 tx=0x7f299000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a3ac2d166e39b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f297cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900120d0 con 0x562afe6f0ef0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2990012e00 con 0x562afe6f0ef0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.957+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298409be80 con 0x562afe6f0ef0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2990012270 con 0x562afe6f0ef0 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 10 client.25811 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562afe369210 con 0x562afe6f0ef0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298409be80 con 0x562afe6f0ef0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f299001c7f0 con 0x562afe6f0ef0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 10 client.25811 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 20 client.25811 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe70a590 0x562afe70c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 4 client.25811 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562afe662120 con 0x562afe70a590 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f2998d19640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe70a590 0x562afe70c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f297cff9640 10 client.25811 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.958+0000 7f2998d19640 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe70a590 0x562afe70c9a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbae8ec193865e61 server_cookie=731e60ce52536585 in_seq=0 out_seq=0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f297cff9640 1 -- 172.21.15.132:0/1527306646 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562afe662120 con 0x562afe70a590 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f297cff9640 10 client.25811 handle_command_reply: tid=0 2024-03-18T23:16:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 client.25811 shutdown 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 msgr2=0x562afe6fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2990012d10 tx=0x7f299000f790 comp rx=0 tx=0).stop 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 shutdown_connections 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe70a590 0x562afe70c9a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f1810 0x562afe6f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f297cff9640 0 client.25811 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 --2- 172.21.15.132:0/1527306646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f0ef0 0x562afe6fa010 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 >> 172.21.15.132:0/1527306646 conn(0x562afe6eb810 msgr2=0x562afe6eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 shutdown_connections 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.959+0000 7f299e36c740 1 -- 172.21.15.132:0/1527306646 wait complete. 2024-03-18T23:16:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.960+0000 7f299e36c740 20 client.25811 trim_cache size 0 max 16384 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 Processor -- start 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 -- start start 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe705fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b6360 con 0x562afe6f0ef0 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b3c30 con 0x562afe6f1810 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299e36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b1aa0 con 0x562afe6fa5e0 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299951a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299951a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58672/0 (socket says 172.21.15.132:58672) 2024-03-18T23:16:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f299951a640 1 -- 172.21.15.132:0/3882250202 learned_addr learned my addr 172.21.15.132:0/3882250202 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f2999d1b640 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.974+0000 7f2998d19640 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe705fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 -- 172.21.15.132:0/3882250202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 msgr2=0x562afe6e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 -- 172.21.15.132:0/3882250202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 msgr2=0x562afe705fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe705fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 -- 172.21.15.132:0/3882250202 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afe435620 con 0x562afe6f0ef0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f299951a640 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f299001cce0 tx=0x7f299000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e474f479e64abb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f297effd640 1 -- 172.21.15.132:0/3882250202 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990024400 con 0x562afe6f0ef0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f297effd640 1 -- 172.21.15.132:0/3882250202 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299001c900 con 0x562afe6f0ef0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.975+0000 7f297effd640 1 -- 172.21.15.132:0/3882250202 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990005740 con 0x562afe6f0ef0 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- 172.21.15.132:0/3882250202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 msgr2=0x562afe70c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f299001cce0 tx=0x7f299000fd10 comp rx=0 tx=0).stop 2024-03-18T23:16:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- 172.21.15.132:0/3882250202 shutdown_connections 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6fa5e0 0x562afe6e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe705fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 --2- 172.21.15.132:0/3882250202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6f0ef0 0x562afe70c980 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- 172.21.15.132:0/3882250202 >> 172.21.15.132:0/3882250202 conn(0x562afe6eb810 msgr2=0x562afe6eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- 172.21.15.132:0/3882250202 shutdown_connections 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- 172.21.15.132:0/3882250202 wait complete. 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 -- start start 2024-03-18T23:16:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.976+0000 7f299e36c740 1 Processor -- start 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 0x562afe702630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b20f0 con 0x562afe6fa5e0 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b3560 con 0x562afe6f1810 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29941b27c0 con 0x562afe6f0ef0 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 10 client.0 _ll_register_callbacks cb 0x7f2990009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299951a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 0x562afe702630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2998d19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58688/0 (socket says 172.21.15.132:58688) 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 -- 172.21.15.132:0/3725247491 learned_addr learned my addr 172.21.15.132:0/3725247491 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 -- 172.21.15.132:0/3725247491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 msgr2=0x562afe702630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 0x562afe702630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 -- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 msgr2=0x562afe6fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 -- 172.21.15.132:0/3725247491 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afe6fe620 con 0x562afe6fa5e0 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2998d19640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299951a640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 0x562afe702630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f2999d1b640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f298c037040 tx=0x7f298c02c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b225a872c4241a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f297cff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c039040 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f298c032a80 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f299001c900 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f298c05c050 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.977+0000 7f299e36c740 10 client.15972 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562afe770600 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299001c900 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f298c03f050 con 0x562afe6fa5e0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 10 client.15972 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 20 client.15972 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe770dd0 0x562afe7731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 4 client.15972 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562afe65c470 con 0x562afe770dd0 2024-03-18T23:16:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299951a640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe770dd0 0x562afe7731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f299951a640 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe770dd0 0x562afe7731f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ed95900239b5eb server_cookie=6a2f76a57b70b181 in_seq=0 out_seq=0 2024-03-18T23:16:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.978+0000 7f297cff9640 10 client.15972 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f297cff9640 1 -- 172.21.15.132:0/3725247491 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562afe65c470 con 0x562afe770dd0 2024-03-18T23:16:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f297cff9640 10 client.15972 handle_command_reply: tid=0 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 client.15972 shutdown 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 msgr2=0x562afe6fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f298c037040 tx=0x7f298c02c180 comp rx=0 tx=0).stop 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 shutdown_connections 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x562afe770dd0 0x562afe7731f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562afe6fa5e0 0x562afe6fe0e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562afe6f1810 0x562afe6fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f297cff9640 0 client.15972 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 --2- 172.21.15.132:0/3725247491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562afe6f0ef0 0x562afe702630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 >> 172.21.15.132:0/3725247491 conn(0x562afe6eb810 msgr2=0x562afe6eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 shutdown_connections 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 1 -- 172.21.15.132:0/3725247491 wait complete. 2024-03-18T23:16:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.985+0000 7f299e36c740 20 client.15972 trim_cache size 0 max 16384 2024-03-18T23:16:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:16:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:16:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:16:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 msgr2=0x7f29800a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 0x7f29800a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f298c004af0 tx=0x7f298c0323c0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 msgr2=0x7f299419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f298400e9c0 tx=0x7f298400ee80 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 shutdown_connections 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f29800a2a10 0x7f29800a4ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29941a2e80 0x7f29941a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2994106b80 0x7f299419eb00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 --2- 172.21.15.132:0/2793978493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29941067a0 0x7f299419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 >> 172.21.15.132:0/2793978493 conn(0x7f2994077560 msgr2=0x7f29940ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 shutdown_connections 2024-03-18T23:16:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:30.986+0000 7f299b77e640 1 -- 172.21.15.132:0/2793978493 wait complete. 2024-03-18T23:16:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:31 smithi196 ceph-mon[67089]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-03-18T23:16:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:31 smithi144 ceph-mon[66781]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-03-18T23:16:31.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:31 smithi132 ceph-mon[32020]: pgmap v223: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-03-18T23:16:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:33 smithi196 ceph-mon[67089]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:33 smithi144 ceph-mon[66781]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:33.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:33 smithi132 ceph-mon[32020]: pgmap v224: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:35 smithi196 ceph-mon[67089]: pgmap v225: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:35.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:35 smithi144 ceph-mon[66781]: pgmap v225: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:35.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:35 smithi132 ceph-mon[32020]: pgmap v225: 97 pgs: 97 active+clean; 4.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:37 smithi196 ceph-mon[67089]: pgmap v226: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:16:37.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:37 smithi144 ceph-mon[66781]: pgmap v226: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:16:37.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:37 smithi132 ceph-mon[32020]: pgmap v226: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:16:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:39 smithi196 ceph-mon[67089]: pgmap v227: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:39.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:39 smithi144 ceph-mon[66781]: pgmap v227: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:39.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:39 smithi132 ceph-mon[32020]: pgmap v227: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:41.667 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:41 smithi132 ceph-mon[32020]: pgmap v228: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:41.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:41 smithi196 ceph-mon[67089]: pgmap v228: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:41.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:41 smithi144 ceph-mon[66781]: pgmap v228: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:16:43.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:43 smithi196 ceph-mon[67089]: pgmap v229: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:43.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:43 smithi144 ceph-mon[66781]: pgmap v229: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:43.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:43 smithi132 ceph-mon[32020]: pgmap v229: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.080+0000 7f8072081640 1 -- 172.21.15.132:0/1024651006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06aa00 msgr2=0x7f806c10e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.080+0000 7f8072081640 1 --2- 172.21.15.132:0/1024651006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06aa00 0x7f806c10e640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8064009ca0 tx=0x7f806402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 -- 172.21.15.132:0/1024651006 shutdown_connections 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 --2- 172.21.15.132:0/1024651006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06aa00 0x7f806c10e640 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 --2- 172.21.15.132:0/1024651006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06a060 0x7f806c06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 --2- 172.21.15.132:0/1024651006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 -- 172.21.15.132:0/1024651006 >> 172.21.15.132:0/1024651006 conn(0x7f806c077440 msgr2=0x7f806c077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 -- 172.21.15.132:0/1024651006 shutdown_connections 2024-03-18T23:16:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 -- 172.21.15.132:0/1024651006 wait complete. 2024-03-18T23:16:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 Processor -- start 2024-03-18T23:16:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.081+0000 7f8072081640 1 -- start start 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 0x7f806c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f806c111be0 con 0x7f806c06a060 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f806c111a60 con 0x7f806c06aa00 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f806c111d60 con 0x7f806c1077f0 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 0x7f806c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41124/0 (socket says 172.21.15.132:41124) 2024-03-18T23:16:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 -- 172.21.15.132:0/4278781706 learned_addr learned my addr 172.21.15.132:0/4278781706 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806bfff640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 -- 172.21.15.132:0/4278781706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 msgr2=0x7f806c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 -- 172.21.15.132:0/4278781706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 msgr2=0x7f806c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 0x7f806c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 -- 172.21.15.132:0/4278781706 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c19f170 con 0x7f806c06a060 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806affd640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 0x7f806c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806bfff640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f806b7fe640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ea60 tx=0x7f805c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8068ff9640 1 -- 172.21.15.132:0/4278781706 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00cde0 con 0x7f806c06a060 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8068ff9640 1 -- 172.21.15.132:0/4278781706 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c0035f0 con 0x7f806c06a060 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.082+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c19f460 con 0x7f806c06a060 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.083+0000 7f8068ff9640 1 -- 172.21.15.132:0/4278781706 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c010670 con 0x7f806c06a060 2024-03-18T23:16:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.083+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c1a6cf0 con 0x7f806c06a060 2024-03-18T23:16:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.085+0000 7f8068ff9640 1 -- 172.21.15.132:0/4278781706 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f805c0037a0 con 0x7f806c06a060 2024-03-18T23:16:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 Processor -- start 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 -- start start 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5590d2e7cbe0 con 0x5590d2f18ef0 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590d2e25960 con 0x5590d2f19810 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8074c6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590d2e78c80 con 0x5590d2f225e0 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8068ff9640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 0x7f80540a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:36196/0 (socket says 172.21.15.132:36196) 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 -- 172.21.15.132:0/1438836865 learned_addr learned my addr 172.21.15.132:0/1438836865 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806bfff640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 0x7f80540a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 -- 172.21.15.132:0/1438836865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 msgr2=0x5590d2f220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 -- 172.21.15.132:0/1438836865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 msgr2=0x5590d2f192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f192d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806affd640 1 -- 172.21.15.132:0/1438836865 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d2e1cd20 con 0x5590d2f225e0 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f8068ff9640 1 -- 172.21.15.132:0/4278781706 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f805c0cf120 con 0x7f806c06a060 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.086+0000 7f806b7fe640 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f806affd640 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80600114f0 tx=0x7f80600119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d987c301da0bd433 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f806bfff640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 0x7f80540a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8064004af0 tx=0x7f806402c0f0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f8051ffb640 1 -- 172.21.15.132:0/1438836865 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806001e070 con 0x5590d2f225e0 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f8051ffb640 1 -- 172.21.15.132:0/1438836865 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806000faa0 con 0x5590d2f225e0 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f8051ffb640 1 -- 172.21.15.132:0/1438836865 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060019b60 con 0x5590d2f225e0 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f8074c6f740 1 -- 172.21.15.132:0/1438836865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 msgr2=0x5590d2f249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.087+0000 7f8074c6f740 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80600114f0 tx=0x7f80600119b0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- 172.21.15.132:0/1438836865 shutdown_connections 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f249d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- 172.21.15.132:0/1438836865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f192d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- 172.21.15.132:0/1438836865 >> 172.21.15.132:0/1438836865 conn(0x5590d2f13810 msgr2=0x5590d2f13bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- 172.21.15.132:0/1438836865 shutdown_connections 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- 172.21.15.132:0/1438836865 wait complete. 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- start start 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 Processor -- start 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 0x5590d2f1f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5590d2f25210 con 0x5590d2f225e0 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590d2e38db0 con 0x5590d2f19810 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5590d2f25390 con 0x5590d2f18ef0 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.088+0000 7f8074c6f740 10 client.0 _ll_register_callbacks cb 0x5590d2e3de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 0x5590d2f1f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:36200/0 (socket says 172.21.15.132:36200) 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 -- 172.21.15.132:0/3645569447 learned_addr learned my addr 172.21.15.132:0/3645569447 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 -- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 msgr2=0x5590d2f1f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f1f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 -- 172.21.15.132:0/3645569447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 msgr2=0x5590d2f1f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 0x5590d2f1f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 -- 172.21.15.132:0/3645569447 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d2e37c60 con 0x5590d2f18ef0 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806affd640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806bfff640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 0x5590d2f1f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f805c01f000 tx=0x7f805c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52776b83f28afb58 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f804affd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:16:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c09b780 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806000faa0 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f805c003b30 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f805c09f8d0 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f8074c6f740 10 client.25828 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.089+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590d2b91210 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806000faa0 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f805c0afb60 con 0x5590d2f18ef0 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 10 client.25828 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 20 client.25828 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f324d0 0x5590d2f34900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 4 client.25828 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590d2e8a120 con 0x5590d2f324d0 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f806affd640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f324d0 0x5590d2f34900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f806affd640 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f324d0 0x5590d2f34900 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3efac476873cd0d server_cookie=28a6cebf9c046eda in_seq=0 out_seq=0 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.090+0000 7f804affd640 10 client.25828 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f804affd640 1 -- 172.21.15.132:0/3645569447 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5590d2e8a120 con 0x5590d2f324d0 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f804affd640 10 client.25828 handle_command_reply: tid=0 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 client.25828 shutdown 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 msgr2=0x5590d2f21f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f805c01f000 tx=0x7f805c009550 comp rx=0 tx=0).stop 2024-03-18T23:16:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 shutdown_connections 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f324d0 0x5590d2f34900 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f225e0 0x5590d2f1f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f804affd640 0 client.25828 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f19810 0x5590d2f1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3645569447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f18ef0 0x5590d2f21f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 >> 172.21.15.132:0/3645569447 conn(0x5590d2f13810 msgr2=0x5590d2f12ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 shutdown_connections 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 1 -- 172.21.15.132:0/3645569447 wait complete. 2024-03-18T23:16:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.091+0000 7f8074c6f740 20 client.25828 trim_cache size 0 max 16384 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 Processor -- start 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 -- start start 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f19810 0x5590d2f0e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 0x5590d2f13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1b1fc0 con 0x5590d2f18ef0 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1af890 con 0x5590d2f225e0 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f8074c6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ad700 con 0x5590d2f19810 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41144/0 (socket says 172.21.15.132:41144) 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f806bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 0x5590d2f13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f806b7fe640 1 -- 172.21.15.132:0/3355191570 learned_addr learned my addr 172.21.15.132:0/3355191570 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.106+0000 7f806affd640 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f19810 0x5590d2f0e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 -- 172.21.15.132:0/3355191570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f19810 msgr2=0x5590d2f0e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f19810 0x5590d2f0e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 -- 172.21.15.132:0/3355191570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 msgr2=0x5590d2f13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 0x5590d2f13bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 -- 172.21.15.132:0/3355191570 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d2f19640 con 0x5590d2f18ef0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806bfff640 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 0x5590d2f13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f805c00d010 tx=0x7f805c003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701e31455bf289d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f80517fa640 1 -- 172.21.15.132:0/3355191570 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c09b780 con 0x5590d2f18ef0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f80517fa640 1 -- 172.21.15.132:0/3355191570 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c0a3f20 con 0x5590d2f18ef0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f80517fa640 1 -- 172.21.15.132:0/3355191570 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c097a20 con 0x5590d2f18ef0 2024-03-18T23:16:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 -- 172.21.15.132:0/3355191570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 msgr2=0x5590d2f2dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f805c00d010 tx=0x7f805c003f30 comp rx=0 tx=0).stop 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 -- 172.21.15.132:0/3355191570 shutdown_connections 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f225e0 0x5590d2f13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f19810 0x5590d2f0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3355191570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f18ef0 0x5590d2f2dfb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.107+0000 7f8074c6f740 1 -- 172.21.15.132:0/3355191570 >> 172.21.15.132:0/3355191570 conn(0x5590d2f13810 msgr2=0x5590d2f12ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- 172.21.15.132:0/3355191570 shutdown_connections 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- 172.21.15.132:0/3355191570 wait complete. 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- start start 2024-03-18T23:16:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 Processor -- start 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f18ef0 0x5590d2f90b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1add50 con 0x5590d2f19810 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1af1c0 con 0x5590d2f18ef0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ae420 con 0x5590d2f225e0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f8074c6f740 10 client.0 _ll_register_callbacks cb 0x7f805c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f806affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f806affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:41152/0 (socket says 172.21.15.132:41152) 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f806affd640 1 -- 172.21.15.132:0/3749485468 learned_addr learned my addr 172.21.15.132:0/3749485468 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f806bfff640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.108+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f18ef0 0x5590d2f90b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 -- 172.21.15.132:0/3749485468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 msgr2=0x5590d2f256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 -- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f18ef0 msgr2=0x5590d2f90b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f18ef0 0x5590d2f90b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 -- 172.21.15.132:0/3749485468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d2c5d620 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f806affd640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8060009250 tx=0x7f806000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f616b057becfd000 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f804affd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806001e040 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f8060016040 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c0a3f20 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f806000f8d0 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f8074c6f740 10 client.15990 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590d2f98180 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.109+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c0a3f20 con 0x5590d2f19810 2024-03-18T23:16:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f806002a580 con 0x5590d2f19810 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 10 client.15990 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 20 client.15990 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f986c0 0x5590d2f9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 4 client.15990 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590d2e84470 con 0x5590d2f986c0 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f986c0 0x5590d2f9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f806b7fe640 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f986c0 0x5590d2f9aae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dca6c1151d1ea6c server_cookie=e2a18796ab16ef9b in_seq=0 out_seq=0 2024-03-18T23:16:44.116 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.110+0000 7f804affd640 10 client.15990 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f804affd640 1 -- 172.21.15.132:0/3749485468 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590d2e84470 con 0x5590d2f986c0 2024-03-18T23:16:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f804affd640 10 client.15990 handle_command_reply: tid=0 2024-03-18T23:16:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 client.15990 shutdown 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 msgr2=0x5590d2f910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8060009250 tx=0x7f806000ff60 comp rx=0 tx=0).stop 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 shutdown_connections 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5590d2f986c0 0x5590d2f9aae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590d2f225e0 0x5590d2f256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590d2f19810 0x5590d2f910a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f804affd640 0 client.15990 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 --2- 172.21.15.132:0/3749485468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590d2f18ef0 0x5590d2f90b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 >> 172.21.15.132:0/3749485468 conn(0x5590d2f13810 msgr2=0x5590d2f12ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 shutdown_connections 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 1 -- 172.21.15.132:0/3749485468 wait complete. 2024-03-18T23:16:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.117+0000 7f8074c6f740 20 client.15990 trim_cache size 0 max 16384 2024-03-18T23:16:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:16:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:16:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:16:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 msgr2=0x7f80540a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 0x7f80540a4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8064004af0 tx=0x7f806402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 msgr2=0x7f806c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ea60 tx=0x7f805c00ef20 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 shutdown_connections 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f80540a2a10 0x7f80540a4ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f806c1077f0 0x7f806c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f806c06aa00 0x7f806c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 --2- 172.21.15.132:0/4278781706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f806c06a060 0x7f806c19a180 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 >> 172.21.15.132:0/4278781706 conn(0x7f806c077440 msgr2=0x7f806c10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 shutdown_connections 2024-03-18T23:16:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:44.118+0000 7f8072081640 1 -- 172.21.15.132:0/4278781706 wait complete. 2024-03-18T23:16:45.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:45 smithi196 ceph-mon[67089]: pgmap v230: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:45.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:45 smithi144 ceph-mon[66781]: pgmap v230: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:45.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:45 smithi132 ceph-mon[32020]: pgmap v230: 97 pgs: 97 active+clean; 4.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:16:47.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:47 smithi196 ceph-mon[67089]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:16:47.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:47 smithi144 ceph-mon[66781]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:47 smithi132 ceph-mon[32020]: pgmap v231: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:16:49.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:49 smithi196 ceph-mon[67089]: pgmap v232: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:49.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:49 smithi144 ceph-mon[66781]: pgmap v232: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:49.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:49 smithi132 ceph-mon[32020]: pgmap v232: 97 pgs: 97 active+clean; 4.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:51.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:51 smithi196 ceph-mon[67089]: pgmap v233: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:51.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:51 smithi144 ceph-mon[66781]: pgmap v233: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:51.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:51 smithi132 ceph-mon[32020]: pgmap v233: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:16:53.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:53 smithi196 ceph-mon[67089]: pgmap v234: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:53.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:53 smithi144 ceph-mon[66781]: pgmap v234: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:53.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:53 smithi132 ceph-mon[32020]: pgmap v234: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:55.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:55 smithi196 ceph-mon[67089]: pgmap v235: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:55.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:55 smithi144 ceph-mon[66781]: pgmap v235: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:55.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:55 smithi132 ceph-mon[32020]: pgmap v235: 97 pgs: 97 active+clean; 4.3 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:16:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 -- 172.21.15.132:0/844430471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40100970 msgr2=0x7f0e40100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 --2- 172.21.15.132:0/844430471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40100970 0x7f0e40100dd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009860 tx=0x7f0e3c02bb70 comp rx=0 tx=0).stop 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 -- 172.21.15.132:0/844430471 shutdown_connections 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 --2- 172.21.15.132:0/844430471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40101310 0x7f0e4010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 --2- 172.21.15.132:0/844430471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40100970 0x7f0e40100dd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 --2- 172.21.15.132:0/844430471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 0x7f0e40105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 -- 172.21.15.132:0/844430471 >> 172.21.15.132:0/844430471 conn(0x7f0e400774f0 msgr2=0x7f0e40077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 -- 172.21.15.132:0/844430471 shutdown_connections 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.213+0000 7f0e480bf640 1 -- 172.21.15.132:0/844430471 wait complete. 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 Processor -- start 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 -- start start 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40101310 0x7f0e40198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 0x7f0e4019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40110bb0 con 0x7f0e40100970 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40110a30 con 0x7f0e401056a0 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e480bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40110d30 con 0x7f0e40101310 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e45e5b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e45e5b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58664/0 (socket says 172.21.15.132:58664) 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e45e5b640 1 -- 172.21.15.132:0/2911243386 learned_addr learned my addr 172.21.15.132:0/2911243386 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e4565a640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40101310 0x7f0e40198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.214+0000 7f0e4665c640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 0x7f0e4019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 -- 172.21.15.132:0/2911243386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40101310 msgr2=0x7f0e40198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40101310 0x7f0e40198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 -- 172.21.15.132:0/2911243386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 msgr2=0x7f0e4019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 0x7f0e4019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 -- 172.21.15.132:0/2911243386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4019d0b0 con 0x7f0e40100970 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400e9f0 tx=0x7f0e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e32ffd640 1 -- 172.21.15.132:0/2911243386 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400cd80 con 0x7f0e40100970 2024-03-18T23:16:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4019d3a0 con 0x7f0e40100970 2024-03-18T23:16:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e32ffd640 1 -- 172.21.15.132:0/2911243386 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e34010910 con 0x7f0e40100970 2024-03-18T23:16:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e32ffd640 1 -- 172.21.15.132:0/2911243386 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340188f0 con 0x7f0e40100970 2024-03-18T23:16:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.215+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e401a4c30 con 0x7f0e40100970 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.217+0000 7f0e32ffd640 1 -- 172.21.15.132:0/2911243386 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f0e34010ac0 con 0x7f0e40100970 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 Processor -- start 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 -- start start 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e32ffd640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 0x7f0e2c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf48c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf48e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf3e6be0 con 0x55dcbf483810 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf38f960 con 0x55dcbf482ef0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf3e2c80 con 0x55dcbf48c5e0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e32ffd640 1 -- 172.21.15.132:0/2911243386 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f0e340cf210 con 0x7f0e40100970 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf48c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4665c640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 0x7f0e2c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e45e5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf48c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58678/0 (socket says 172.21.15.132:58678) 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4565a640 1 -- 172.21.15.132:0/1424763236 learned_addr learned my addr 172.21.15.132:0/1424763236 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.218+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf48e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1424763236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 msgr2=0x55dcbf48e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf48e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1424763236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 msgr2=0x55dcbf48c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf48c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1424763236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbf386d20 con 0x55dcbf482ef0 2024-03-18T23:16:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e4665c640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 0x7f0e2c0a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e38005fd0 tx=0x7f0e380099c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4832d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e340cc960 tx=0x7f0e34097cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b43e014bd71710 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e2bfff640 1 -- 172.21.15.132:0/1424763236 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34084e60 con 0x55dcbf482ef0 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e2bfff640 1 -- 172.21.15.132:0/1424763236 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3409b920 con 0x55dcbf482ef0 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.219+0000 7f0e2bfff640 1 -- 172.21.15.132:0/1424763236 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34097f90 con 0x55dcbf482ef0 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1424763236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 msgr2=0x55dcbf4832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4832d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e340cc960 tx=0x7f0e34097cf0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1424763236 shutdown_connections 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf48e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf48c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1424763236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4832d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1424763236 >> 172.21.15.132:0/1424763236 conn(0x55dcbf47d810 msgr2=0x55dcbf47dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1424763236 shutdown_connections 2024-03-18T23:16:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1424763236 wait complete. 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 -- start start 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.220+0000 7f0e4ad88740 1 Processor -- start 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf48c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf489b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf48f150 con 0x55dcbf483810 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf3a2db0 con 0x55dcbf482ef0 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dcbf48f2d0 con 0x55dcbf48c5e0 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4ad88740 10 client.0 _ll_register_callbacks cb 0x55dcbf3a7e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58686/0 (socket says 172.21.15.132:58686) 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 -- 172.21.15.132:0/882494786 learned_addr learned my addr 172.21.15.132:0/882494786 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4665c640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf489b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf48c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 -- 172.21.15.132:0/882494786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 msgr2=0x55dcbf489b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf489b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 -- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 msgr2=0x55dcbf48c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf48c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 -- 172.21.15.132:0/882494786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbf3a1c60 con 0x55dcbf483810 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4665c640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf489b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf48c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e4565a640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c02c180 tx=0x7f0e3c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4661256e0e3d6a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.221+0000 7f0e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c043070 con 0x55dcbf483810 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f0e3c004450 con 0x55dcbf483810 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3409b920 con 0x55dcbf483810 2024-03-18T23:16:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e3c04f050 con 0x55dcbf483810 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 10 client.16002 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dcbf0fb210 con 0x55dcbf483810 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3409b920 con 0x55dcbf483810 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e3c032640 con 0x55dcbf483810 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 10 client.16002 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 20 client.16002 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf49c490 0x55dcbf49e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 4 client.16002 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcbf3f4120 con 0x55dcbf49c490 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.222+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf49c490 0x55dcbf49e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e297fa640 10 client.16002 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf49c490 0x55dcbf49e8a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abe357e105d5ab1 server_cookie=fdcb1de3cd06265f in_seq=0 out_seq=0 2024-03-18T23:16:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e297fa640 1 -- 172.21.15.132:0/882494786 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dcbf3f4120 con 0x55dcbf49c490 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e297fa640 10 client.16002 handle_command_reply: tid=0 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e4ad88740 1 client.16002 shutdown 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 msgr2=0x55dcbf4893e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.223+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c02c180 tx=0x7f0e3c009540 comp rx=0 tx=0).stop 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 shutdown_connections 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf49c490 0x55dcbf49e8a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf48c5e0 0x55dcbf489b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf483810 0x55dcbf4893e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/882494786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf48c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e297fa640 0 client.16002 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 >> 172.21.15.132:0/882494786 conn(0x55dcbf47d810 msgr2=0x55dcbf47cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 shutdown_connections 2024-03-18T23:16:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 1 -- 172.21.15.132:0/882494786 wait complete. 2024-03-18T23:16:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.224+0000 7f0e4ad88740 20 client.16002 trim_cache size 0 max 16384 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 Processor -- start 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 -- start start 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf497ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 0x55dcbf478940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401afc50 con 0x55dcbf482ef0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401ad520 con 0x55dcbf48c5e0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.238+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401ab390 con 0x55dcbf483810 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58696/0 (socket says 172.21.15.132:58696) 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1716501625 learned_addr learned my addr 172.21.15.132:0/1716501625 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e4565a640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf497ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 0x55dcbf478940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1716501625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 msgr2=0x55dcbf497ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf497ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1716501625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 msgr2=0x55dcbf478940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 0x55dcbf478940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 -- 172.21.15.132:0/1716501625 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbf1c7620 con 0x55dcbf482ef0 2024-03-18T23:16:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e4565a640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf497ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 0x55dcbf478940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ccf60 tx=0x7f0e3400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b14f947afa054a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e2b7fe640 1 -- 172.21.15.132:0/1716501625 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34084e60 con 0x55dcbf482ef0 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e2b7fe640 1 -- 172.21.15.132:0/1716501625 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e340a4010 con 0x55dcbf482ef0 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.239+0000 7f0e2b7fe640 1 -- 172.21.15.132:0/1716501625 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3409ad70 con 0x55dcbf482ef0 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1716501625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 msgr2=0x55dcbf494900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ccf60 tx=0x7f0e3400cf20 comp rx=0 tx=0).stop 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1716501625 shutdown_connections 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf48c5e0 0x55dcbf478940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf497ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1716501625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf482ef0 0x55dcbf494900 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1716501625 >> 172.21.15.132:0/1716501625 conn(0x55dcbf47d810 msgr2=0x55dcbf47cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1716501625 shutdown_connections 2024-03-18T23:16:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1716501625 wait complete. 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 -- start start 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.240+0000 7f0e4ad88740 1 Processor -- start 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 0x55dcbf4fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401ab9e0 con 0x55dcbf48c5e0 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401ace50 con 0x55dcbf482ef0 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e401ac0b0 con 0x55dcbf483810 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4ad88740 10 client.0 _ll_register_callbacks cb 0x7f0e34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 0x55dcbf4fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4565a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:51072/0 (socket says 172.21.15.132:51072) 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4565a640 1 -- 172.21.15.132:0/1874304768 learned_addr learned my addr 172.21.15.132:0/1874304768 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 -- 172.21.15.132:0/1874304768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 msgr2=0x55dcbf4face0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 -- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 msgr2=0x55dcbf4fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 -- 172.21.15.132:0/1874304768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbf505a70 con 0x55dcbf48c5e0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4565a640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e4665c640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 0x55dcbf4fb220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55dcbf48a430 tx=0x7f0e38004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce952a1c8bae2621 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.241+0000 7f0e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3802d020 con 0x55dcbf48c5e0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f0e38017070 con 0x55dcbf48c5e0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e340a4010 con 0x55dcbf48c5e0 2024-03-18T23:16:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e38038050 con 0x55dcbf48c5e0 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 10 client.16008 fetch_fsmap learned FSMap version 24 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dcbf506d80 con 0x55dcbf48c5e0 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e340a4010 con 0x55dcbf48c5e0 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e3800b040 con 0x55dcbf48c5e0 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 10 client.16008 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 20 client.16008 populate_metadata read hostname 'smithi132' 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf507550 0x55dcbf509970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 4 client.16008 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcbf3ee470 con 0x55dcbf507550 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.242+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf507550 0x55dcbf509970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.243+0000 7f0e297fa640 10 client.16008 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.243+0000 7f0e45e5b640 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf507550 0x55dcbf509970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ac90cba9fbba2b server_cookie=caf44273a89fb08e in_seq=0 out_seq=0 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.249+0000 7f0e297fa640 1 -- 172.21.15.132:0/1874304768 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcbf3ee470 con 0x55dcbf507550 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.249+0000 7f0e297fa640 10 client.16008 handle_command_reply: tid=0 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.249+0000 7f0e4ad88740 1 client.16008 shutdown 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 msgr2=0x55dcbf4fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 0x55dcbf4fb220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55dcbf48a430 tx=0x7f0e38004ab0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 shutdown_connections 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dcbf507550 0x55dcbf509970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dcbf48c5e0 0x55dcbf4fb220 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dcbf483810 0x55dcbf4face0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e297fa640 0 client.16008 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:16:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 --2- 172.21.15.132:0/1874304768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dcbf482ef0 0x55dcbf4fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 >> 172.21.15.132:0/1874304768 conn(0x55dcbf47d810 msgr2=0x55dcbf47cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 shutdown_connections 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 1 -- 172.21.15.132:0/1874304768 wait complete. 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e4ad88740 20 client.16008 trim_cache size 0 max 16384 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:16:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 msgr2=0x7f0e2c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 0x7f0e2c0a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e38005fd0 tx=0x7f0e380099c0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 msgr2=0x7f0e401980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400e9f0 tx=0x7f0e3400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 shutdown_connections 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f0e2c0a2a10 0x7f0e2c0a4ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e401056a0 0x7f0e4019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40101310 0x7f0e40198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 --2- 172.21.15.132:0/2911243386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e40100970 0x7f0e401980c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:16:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 >> 172.21.15.132:0/2911243386 conn(0x7f0e400774f0 msgr2=0x7f0e4006a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:16:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 shutdown_connections 2024-03-18T23:16:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:16:57.250+0000 7f0e480bf640 1 -- 172.21.15.132:0/2911243386 wait complete. 2024-03-18T23:16:57.660 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:57 smithi144 ceph-mon[66781]: pgmap v236: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:57.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:57 smithi196 ceph-mon[67089]: pgmap v236: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:57.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:57 smithi132 ceph-mon[32020]: pgmap v236: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:16:59.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:16:59 smithi196 ceph-mon[67089]: pgmap v237: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:16:59.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:16:59 smithi144 ceph-mon[66781]: pgmap v237: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:16:59.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:16:59 smithi132 ceph-mon[32020]: pgmap v237: 97 pgs: 97 active+clean; 4.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:17:01.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:01 smithi196 ceph-mon[67089]: pgmap v238: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:17:01.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:01 smithi144 ceph-mon[66781]: pgmap v238: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:17:01.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:01 smithi132 ceph-mon[32020]: pgmap v238: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:17:03.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:03 smithi196 ceph-mon[67089]: pgmap v239: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:17:03.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:03 smithi144 ceph-mon[66781]: pgmap v239: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:17:03.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:03 smithi132 ceph-mon[32020]: pgmap v239: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:17:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:05 smithi196 ceph-mon[67089]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:17:05.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:05 smithi196 ceph-mon[67089]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:05 smithi144 ceph-mon[66781]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:17:05.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:05 smithi144 ceph-mon[66781]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:05 smithi132 ceph-mon[32020]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T23:17:05.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:05 smithi132 ceph-mon[32020]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:07.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:07 smithi196 ceph-mon[67089]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:07.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:07 smithi144 ceph-mon[66781]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:07.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:07 smithi132 ceph-mon[32020]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:17:10.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:09 smithi196 ceph-mon[67089]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:17:10.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:09 smithi196 ceph-mon[67089]: pgmap v244: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 -- 172.21.15.132:0/1577328583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04068ea0 msgr2=0x7fec04069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 --2- 172.21.15.132:0/1577328583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04068ea0 0x7fec04069300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febf8009930 tx=0x7febf802bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 -- 172.21.15.132:0/1577328583 shutdown_connections 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 --2- 172.21.15.132:0/1577328583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04069840 0x7fec0410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 --2- 172.21.15.132:0/1577328583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec04068ea0 0x7fec04069300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 --2- 172.21.15.132:0/1577328583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec041078f0 0x7fec04107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 -- 172.21.15.132:0/1577328583 >> 172.21.15.132:0/1577328583 conn(0x7fec04077570 msgr2=0x7fec04077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 -- 172.21.15.132:0/1577328583 shutdown_connections 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 -- 172.21.15.132:0/1577328583 wait complete. 2024-03-18T23:17:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.345+0000 7fec09c1f640 1 Processor -- start 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 -- start start 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 0x7fec04198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 0x7fec041985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111ba0 con 0x7fec04068ea0 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111a20 con 0x7fec041078f0 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec09c1f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111d20 con 0x7fec04069840 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 0x7fec04198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 0x7fec041985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:39430/0 (socket says 172.21.15.132:39430) 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 0x7fec041985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:43960/0 (socket says 172.21.15.132:43960) 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 0x7fec04198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53760/0 (socket says 172.21.15.132:53760) 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 -- 172.21.15.132:0/3551808172 learned_addr learned my addr 172.21.15.132:0/3551808172 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 -- 172.21.15.132:0/3551808172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 msgr2=0x7fec041985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 0x7fec041985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 -- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 msgr2=0x7fec04198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 0x7fec04198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 -- 172.21.15.132:0/3551808172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0419d000 con 0x7fec041078f0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec03fff640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7febfc00ea60 tx=0x7febfc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.346+0000 7fec00ff9640 1 -- 172.21.15.132:0/3551808172 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc00ce40 con 0x7fec041078f0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.347+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0419d2f0 con 0x7fec041078f0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.347+0000 7fec00ff9640 1 -- 172.21.15.132:0/3551808172 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febfc005020 con 0x7fec041078f0 2024-03-18T23:17:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.347+0000 7fec00ff9640 1 -- 172.21.15.132:0/3551808172 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc010690 con 0x7fec041078f0 2024-03-18T23:17:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.347+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a4b80 con 0x7fec041078f0 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec00ff9640 1 -- 172.21.15.132:0/3551808172 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7febfc005520 con 0x7fec041078f0 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 Processor -- start 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 -- start start 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec00ff9640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 0x7febec0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 0x5609a77fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a78094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec02ffd640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 0x7febec0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5609a775fbe0 con 0x5609a78099f0 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609a7708960 con 0x5609a77fc330 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec0c805740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609a775bc80 con 0x5609a77fba10 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec00ff9640 1 -- 172.21.15.132:0/3551808172 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7febfc0cf330 con 0x7fec041078f0 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a78094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec03fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec037fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 0x5609a77fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a78094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:39432/0 (socket says 172.21.15.132:39432) 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.349+0000 7fec02ffd640 1 -- 172.21.15.132:0/4175132990 learned_addr learned my addr 172.21.15.132:0/4175132990 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 -- 172.21.15.132:0/4175132990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 msgr2=0x5609a78094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a78094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 -- 172.21.15.132:0/4175132990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 msgr2=0x5609a780bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 -- 172.21.15.132:0/4175132990 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a76ffd20 con 0x5609a77fba10 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec03fff640 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec02ffd640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 0x7febec0a4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febf8006010 tx=0x7febf802c650 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7fec037fe640 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 0x5609a77fbdf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c20 tx=0x7febf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909ab1fd75ce8ba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7febe9ffb640 1 -- 172.21.15.132:0/4175132990 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4004050 con 0x5609a77fba10 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7febe9ffb640 1 -- 172.21.15.132:0/4175132990 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf4005500 con 0x5609a77fba10 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.350+0000 7febe9ffb640 1 -- 172.21.15.132:0/4175132990 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4004e00 con 0x5609a77fba10 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- 172.21.15.132:0/4175132990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 msgr2=0x5609a77fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 0x5609a77fbdf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c20 tx=0x7febf4011810 comp rx=0 tx=0).stop 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- 172.21.15.132:0/4175132990 shutdown_connections 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a78094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 --2- 172.21.15.132:0/4175132990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fba10 0x5609a77fbdf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- 172.21.15.132:0/4175132990 >> 172.21.15.132:0/4175132990 conn(0x5609a77f6810 msgr2=0x5609a77f6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- 172.21.15.132:0/4175132990 shutdown_connections 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- 172.21.15.132:0/4175132990 wait complete. 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 -- start start 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.351+0000 7fec0c805740 1 Processor -- start 2024-03-18T23:17:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a7806b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x5609a780c5b0 con 0x5609a77fba10 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609a771bdb0 con 0x5609a77fc330 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609a780c730 con 0x5609a78099f0 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec0c805740 10 client.0 _ll_register_callbacks cb 0x5609a7720e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec02ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:39444/0 (socket says 172.21.15.132:39444) 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53776/0 (socket says 172.21.15.132:53776) 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a7806b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec02ffd640 1 -- 172.21.15.132:0/3350094290 learned_addr learned my addr 172.21.15.132:0/3350094290 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 -- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 msgr2=0x5609a7806640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 -- 172.21.15.132:0/3350094290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 msgr2=0x5609a78092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 -- 172.21.15.132:0/3350094290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a771ac60 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec02ffd640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec037fe640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7fec03fff640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a7806b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febfc0cca80 tx=0x7febfc09ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57c2bb0e069a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7febe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.352+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b3020 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf4005500 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7febfc098000 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febfc0a3ca0 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 10 client.25864 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5609a7474210 con 0x5609a78099f0 2024-03-18T23:17:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf4005500 con 0x5609a78099f0 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7febfc09ade0 con 0x5609a78099f0 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 10 client.25864 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 20 client.25864 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7819860 0x5609a781bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 4 client.25864 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a776d120 con 0x5609a7819860 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.353+0000 7fec037fe640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7819860 0x5609a781bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.354+0000 7fec037fe640 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7819860 0x5609a781bc50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1f801b3fac8cbb server_cookie=b3333b420424ac15 in_seq=0 out_seq=0 2024-03-18T23:17:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.354+0000 7febe2ffd640 10 client.25864 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:09 smithi144 ceph-mon[66781]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:17:10.360 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:09 smithi144 ceph-mon[66781]: pgmap v244: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7febe2ffd640 1 -- 172.21.15.132:0/3350094290 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5609a776d120 con 0x5609a7819860 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7febe2ffd640 10 client.25864 handle_command_reply: tid=0 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 client.25864 shutdown 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 msgr2=0x5609a7806b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a7806b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febfc0cca80 tx=0x7febfc09ba50 comp rx=0 tx=0).stop 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 shutdown_connections 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7819860 0x5609a781bc50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a7806b80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7febe2ffd640 0 client.25864 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7806640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 --2- 172.21.15.132:0/3350094290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a78092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 >> 172.21.15.132:0/3350094290 conn(0x5609a77f6810 msgr2=0x5609a77f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 shutdown_connections 2024-03-18T23:17:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 1 -- 172.21.15.132:0/3350094290 wait complete. 2024-03-18T23:17:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.355+0000 7fec0c805740 20 client.25864 trim_cache size 0 max 16384 2024-03-18T23:17:10.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:09 smithi132 ceph-mon[32020]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T23:17:10.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:09 smithi132 ceph-mon[32020]: pgmap v244: 97 pgs: 97 active+clean; 4.4 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:17:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 Processor -- start 2024-03-18T23:17:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 -- start start 2024-03-18T23:17:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7821540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a78119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fec041adb60 con 0x5609a77fba10 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec041ab430 con 0x5609a77fc330 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec0c805740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a92a0 con 0x5609a78099f0 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53784/0 (socket says 172.21.15.132:53784) 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 -- 172.21.15.132:0/2388807383 learned_addr learned my addr 172.21.15.132:0/2388807383 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec03fff640 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a78119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec02ffd640 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7821540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 -- 172.21.15.132:0/2388807383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 msgr2=0x5609a78119b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a78119b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 -- 172.21.15.132:0/2388807383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 msgr2=0x5609a7821540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7821540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 -- 172.21.15.132:0/2388807383 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a7540620 con 0x5609a77fba10 2024-03-18T23:17:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.370+0000 7fec037fe640 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7febf4000c00 tx=0x7febf4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caad741c58918620 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7febe97fa640 1 -- 172.21.15.132:0/2388807383 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4031070 con 0x5609a77fba10 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7febe97fa640 1 -- 172.21.15.132:0/2388807383 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf401f040 con 0x5609a77fba10 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7febe97fa640 1 -- 172.21.15.132:0/2388807383 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf401e040 con 0x5609a77fba10 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 -- 172.21.15.132:0/2388807383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 msgr2=0x5609a77f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7febf4000c00 tx=0x7febf4009680 comp rx=0 tx=0).stop 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 -- 172.21.15.132:0/2388807383 shutdown_connections 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a78099f0 0x5609a78119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fc330 0x5609a7821540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 --2- 172.21.15.132:0/2388807383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a77fba10 0x5609a77f5930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 -- 172.21.15.132:0/2388807383 >> 172.21.15.132:0/2388807383 conn(0x5609a77f6810 msgr2=0x5609a77f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 -- 172.21.15.132:0/2388807383 shutdown_connections 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.371+0000 7fec0c805740 1 -- 172.21.15.132:0/2388807383 wait complete. 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 -- start start 2024-03-18T23:17:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 Processor -- start 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fba10 0x5609a780d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fc330 0x5609a780d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a98f0 con 0x5609a78099f0 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec041aad60 con 0x5609a77fba10 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec041a9fc0 con 0x5609a77fc330 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec0c805740 10 client.0 _ll_register_callbacks cb 0x7febfc09b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:53788/0 (socket says 172.21.15.132:53788) 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 -- 172.21.15.132:0/1345243569 learned_addr learned my addr 172.21.15.132:0/1345243569 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fba10 0x5609a780d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec02ffd640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fc330 0x5609a780d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 -- 172.21.15.132:0/1345243569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fc330 msgr2=0x5609a780d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fc330 0x5609a780d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 -- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fba10 msgr2=0x5609a780d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.372+0000 7fec03fff640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fba10 0x5609a780d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec03fff640 1 -- 172.21.15.132:0/1345243569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a787ecd0 con 0x5609a78099f0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec03fff640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7febfc0049d0 tx=0x7febfc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21bdc269db80a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b3040 con 0x5609a78099f0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7febfc098000 con 0x5609a78099f0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf401f040 con 0x5609a78099f0 2024-03-18T23:17:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7febfc09fae0 con 0x5609a78099f0 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 10 client.16026 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5609a787ffc0 con 0x5609a78099f0 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf401f040 con 0x5609a78099f0 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7febfc0a0a00 con 0x5609a78099f0 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 10 client.16026 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 20 client.16026 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7880520 0x5609a7882940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 4 client.16026 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.373+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609a7767470 con 0x5609a7880520 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.374+0000 7fec037fe640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7880520 0x5609a7882940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.374+0000 7febe2ffd640 10 client.16026 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.374+0000 7fec037fe640 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7880520 0x5609a7882940 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbd1ae5dd6edf3c server_cookie=78b1d80364f3ff8f in_seq=0 out_seq=0 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7febe2ffd640 1 -- 172.21.15.132:0/1345243569 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609a7767470 con 0x5609a7880520 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7febe2ffd640 10 client.16026 handle_command_reply: tid=0 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 client.16026 shutdown 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 msgr2=0x5609a780dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7febfc0049d0 tx=0x7febfc0096c0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 shutdown_connections 2024-03-18T23:17:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5609a7880520 0x5609a7882940 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609a78099f0 0x5609a780dee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7febe2ffd640 0 client.16026 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a77fc330 0x5609a780d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 --2- 172.21.15.132:0/1345243569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609a77fba10 0x5609a780d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 >> 172.21.15.132:0/1345243569 conn(0x5609a77f6810 msgr2=0x5609a77f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 shutdown_connections 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 1 -- 172.21.15.132:0/1345243569 wait complete. 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.381+0000 7fec0c805740 20 client.16026 trim_cache size 0 max 16384 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:17:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:17:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:17:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 msgr2=0x7febec0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 0x7febec0a4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7febf8006010 tx=0x7febf802c650 comp rx=0 tx=0).stop 2024-03-18T23:17:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 msgr2=0x7fec0419c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7febfc00ea60 tx=0x7febfc00ef20 comp rx=0 tx=0).stop 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 shutdown_connections 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7febec0a2a10 0x7febec0a4ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec041078f0 0x7fec0419c8d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec04069840 0x7fec041985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 --2- 172.21.15.132:0/3551808172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec04068ea0 0x7fec04198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 >> 172.21.15.132:0/3551808172 conn(0x7fec04077570 msgr2=0x7fec040fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 shutdown_connections 2024-03-18T23:17:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:10.382+0000 7fec09c1f640 1 -- 172.21.15.132:0/3551808172 wait complete. 2024-03-18T23:17:11.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:11 smithi196 ceph-mon[67089]: pgmap v245: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:17:11.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:11 smithi144 ceph-mon[66781]: pgmap v245: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:17:11.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:11 smithi132 ceph-mon[32020]: pgmap v245: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:17:13.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:13 smithi196 ceph-mon[67089]: pgmap v246: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:17:13.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:13 smithi144 ceph-mon[66781]: pgmap v246: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:17:13.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:13 smithi132 ceph-mon[32020]: pgmap v246: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:375/600: open fd = 3 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:376/600: open fd = 3 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:377/600: open fd = 3 2024-03-18T23:17:14.831 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:378/600: open fd = 3 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:379/600: open fd = 3 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:380/600: open fd = 3 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.832 INFO:tasks.workunit.client.0.smithi132.stdout:381/600: open fd = 3 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:382/600: open fd = 3 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:383/600: open fd = 3 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:384/600: open fd = 3 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.833 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:385/600: open fd = 3 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:386/600: open fd = 3 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:387/600: open fd = 3 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.834 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:388/600: open fd = 3 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:389/600: open fd = 3 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:390/600: open fd = 3 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:391/600: open fd = 3 2024-03-18T23:17:14.835 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:392/600: open fd = 3 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:393/600: open fd = 3 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:394/600: open fd = 3 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.836 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:395/600: open fd = 3 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:396/600: open fd = 3 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:397/600: open fd = 3 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:398/600: open fd = 3 2024-03-18T23:17:14.837 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:399/600: open fd = 3 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:400/600: open fd = 3 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:401/600: open fd = 3 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.838 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:402/600: open fd = 3 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:403/600: open fd = 3 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:404/600: open fd = 3 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.839 INFO:tasks.workunit.client.0.smithi132.stdout:405/600: open fd = 3 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:406/600: open fd = 3 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:407/600: open fd = 3 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:408/600: open fd = 3 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.840 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:409/600: open fd = 3 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:410/600: open fd = 3 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:411/600: open fd = 3 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.841 INFO:tasks.workunit.client.0.smithi132.stdout:412/600: open fd = 3 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:413/600: open fd = 3 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:414/600: open fd = 3 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:415/600: open fd = 3 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.842 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:416/600: open fd = 3 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:417/600: open fd = 3 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:418/600: open fd = 3 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:419/600: open fd = 3 2024-03-18T23:17:14.843 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:420/600: open fd = 3 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:421/600: open fd = 3 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:422/600: open fd = 3 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.844 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:423/600: open fd = 3 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:424/600: open fd = 3 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:425/600: open fd = 3 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:426/600: open fd = 3 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.845 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:427/600: open fd = 3 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:428/600: open fd = 3 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:429/600: open fd = 3 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.846 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:430/600: open fd = 3 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:431/600: open fd = 3 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:432/600: open fd = 3 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:433/600: open fd = 3 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.847 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:434/600: open fd = 3 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:435/600: open fd = 3 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:436/600: open fd = 3 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:17:14.848 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:17:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:15 smithi196 ceph-mon[67089]: pgmap v247: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:15.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:15 smithi144 ceph-mon[66781]: pgmap v247: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:15.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:15 smithi132 ceph-mon[32020]: pgmap v247: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:17.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:17 smithi196 ceph-mon[67089]: pgmap v248: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:17.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:17 smithi144 ceph-mon[66781]: pgmap v248: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:17.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:17 smithi132 ceph-mon[32020]: pgmap v248: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:17:19.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:19 smithi196 ceph-mon[67089]: pgmap v249: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:17:19.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:19 smithi144 ceph-mon[66781]: pgmap v249: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:17:19.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:19 smithi132 ceph-mon[32020]: pgmap v249: 97 pgs: 97 active+clean; 4.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:17:21.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:21 smithi196 ceph-mon[67089]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:21.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:21 smithi144 ceph-mon[66781]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:21 smithi132 ceph-mon[32020]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 -- 172.21.15.132:0/800239628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8741006f0 msgr2=0x7ff874100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 --2- 172.21.15.132:0/800239628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8741006f0 0x7ff874100b70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff864009930 tx=0x7ff86402bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 -- 172.21.15.132:0/800239628 shutdown_connections 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 --2- 172.21.15.132:0/800239628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741010b0 0x7ff87410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 --2- 172.21.15.132:0/800239628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8741006f0 0x7ff874100b70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 --2- 172.21.15.132:0/800239628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff874106700 0x7ff874106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 -- 172.21.15.132:0/800239628 >> 172.21.15.132:0/800239628 conn(0x7ff874077850 msgr2=0x7ff874077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 -- 172.21.15.132:0/800239628 shutdown_connections 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.478+0000 7ff87aadc640 1 -- 172.21.15.132:0/800239628 wait complete. 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 Processor -- start 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 -- start start 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 0x7ff874199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 0x7ff8741a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110ad0 con 0x7ff874106700 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110950 con 0x7ff8741010b0 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff87aadc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110c50 con 0x7ff8741006f0 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff878878640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff879079640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 0x7ff8741a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff878878640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:35364/0 (socket says 172.21.15.132:35364) 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 0x7ff874199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.479+0000 7ff878878640 1 -- 172.21.15.132:0/2561020737 learned_addr learned my addr 172.21.15.132:0/2561020737 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 -- 172.21.15.132:0/2561020737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 msgr2=0x7ff874198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 -- 172.21.15.132:0/2561020737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 msgr2=0x7ff874199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 0x7ff874199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 -- 172.21.15.132:0/2561020737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8741a43f0 con 0x7ff874106700 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff878878640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff86bfff640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 0x7ff874199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff879079640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 0x7ff8741a3eb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ec50 tx=0x7ff86c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff869ffb640 1 -- 172.21.15.132:0/2561020737 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003b70 con 0x7ff874106700 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff869ffb640 1 -- 172.21.15.132:0/2561020737 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c003d10 con 0x7ff874106700 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741a4620 con 0x7ff874106700 2024-03-18T23:17:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff869ffb640 1 -- 172.21.15.132:0/2561020737 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c010890 con 0x7ff874106700 2024-03-18T23:17:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.480+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a4af0 con 0x7ff874106700 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.482+0000 7ff869ffb640 1 -- 172.21.15.132:0/2561020737 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7ff86c003fe0 con 0x7ff874106700 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff869ffb640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 0x7ff85c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 Processor -- start 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff878878640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 0x7ff85c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff869ffb640 1 -- 172.21.15.132:0/2561020737 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff86c0cf3f0 con 0x7ff874106700 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 -- start start 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 0x56149a9ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149a912be0 con 0x56149a9b85e0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56149a8bb960 con 0x56149a9af810 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56149a90ec80 con 0x56149a9aeef0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff878878640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 0x7ff85c0a4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff870011440 tx=0x7ff870005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff879079640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 0x56149a9ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff879079640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 0x56149a9ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40302/0 (socket says 172.21.15.132:40302) 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff879079640 1 -- 172.21.15.132:0/68857142 learned_addr learned my addr 172.21.15.132:0/68857142 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.483+0000 7ff878878640 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 -- 172.21.15.132:0/68857142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 msgr2=0x56149a9af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 -- 172.21.15.132:0/68857142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 msgr2=0x56149a9ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 0x56149a9ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 -- 172.21.15.132:0/68857142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149a8b2d20 con 0x56149a9af810 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff86bfff640 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9b80a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff864009230 tx=0x7ff864009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27eb82178da64d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff85affd640 1 -- 172.21.15.132:0/68857142 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864042070 con 0x56149a9af810 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff85affd640 1 -- 172.21.15.132:0/68857142 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864004a70 con 0x56149a9af810 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.484+0000 7ff85affd640 1 -- 172.21.15.132:0/68857142 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864005190 con 0x56149a9af810 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- 172.21.15.132:0/68857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 msgr2=0x56149a9b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9b80a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff864009230 tx=0x7ff864009630 comp rx=0 tx=0).stop 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- 172.21.15.132:0/68857142 shutdown_connections 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9b85e0 0x56149a9ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9b80a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/68857142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- 172.21.15.132:0/68857142 >> 172.21.15.132:0/68857142 conn(0x56149a9a9810 msgr2=0x56149a9a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- 172.21.15.132:0/68857142 shutdown_connections 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- 172.21.15.132:0/68857142 wait complete. 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 -- start start 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.485+0000 7ff87d7a1740 1 Processor -- start 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56149a9bb1b0 con 0x56149a9af810 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56149a8cedb0 con 0x56149a9aeef0 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56149a9bb330 con 0x56149a9b85e0 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff87d7a1740 10 client.0 _ll_register_callbacks cb 0x56149a8d3e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:33784/0 (socket says 172.21.15.132:33784) 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40318/0 (socket says 172.21.15.132:40318) 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 -- 172.21.15.132:0/2600031120 learned_addr learned my addr 172.21.15.132:0/2600031120 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff879079640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 -- 172.21.15.132:0/2600031120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 msgr2=0x56149a9b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 -- 172.21.15.132:0/2600031120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 msgr2=0x56149a9b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 -- 172.21.15.132:0/2600031120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149a8cdc60 con 0x56149a9aeef0 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff879079640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff86bfff640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff878878640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff87000fc30 tx=0x7ff870013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb14a13822b2c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff853fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.486+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87002c060 con 0x56149a9aeef0 2024-03-18T23:17:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff864004a70 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff8700273f0 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff870027630 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 10 client.25883 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56149a627210 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864004a70 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff870026070 con 0x56149a9aeef0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 10 client.25883 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 10 client.25883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 20 client.25883 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149a9c84a0 0x56149a9ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 4 client.25883 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56149a920120 con 0x56149a9c84a0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.487+0000 7ff86bfff640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149a9c84a0 0x56149a9ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.488+0000 7ff853fff640 10 client.25883 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.488+0000 7ff86bfff640 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149a9c84a0 0x56149a9ca890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9421aa520c1d3af server_cookie=d360b853184b2a4d in_seq=0 out_seq=0 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff853fff640 1 -- 172.21.15.132:0/2600031120 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56149a920120 con 0x56149a9c84a0 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff853fff640 10 client.25883 handle_command_reply: tid=0 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 client.25883 shutdown 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 msgr2=0x56149a9b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff87000fc30 tx=0x7ff870013040 comp rx=0 tx=0).stop 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 shutdown_connections 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149a9c84a0 0x56149a9ca890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff853fff640 0 client.25883 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/2600031120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9aeef0 0x56149a9b7f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 >> 172.21.15.132:0/2600031120 conn(0x56149a9a9810 msgr2=0x56149a9a8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 shutdown_connections 2024-03-18T23:17:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 1 -- 172.21.15.132:0/2600031120 wait complete. 2024-03-18T23:17:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.489+0000 7ff87d7a1740 20 client.25883 trim_cache size 0 max 16384 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 Processor -- start 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 -- start start 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9b85e0 0x56149a9b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741afc30 con 0x56149a9af810 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ad500 con 0x56149a9b85e0 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ab370 con 0x56149a9aeef0 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40334/0 (socket says 172.21.15.132:40334) 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 -- 172.21.15.132:0/3363471765 learned_addr learned my addr 172.21.15.132:0/3363471765 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff878878640 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff879079640 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9b85e0 0x56149a9b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 -- 172.21.15.132:0/3363471765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 msgr2=0x56149a9c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 -- 172.21.15.132:0/3363471765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9b85e0 msgr2=0x56149a9b7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9b85e0 0x56149a9b7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 -- 172.21.15.132:0/3363471765 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149a6f3620 con 0x56149a9af810 2024-03-18T23:17:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.504+0000 7ff86bfff640 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff86402cfd0 tx=0x7ff864009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeb1e73228fbc6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff85a7fc640 1 -- 172.21.15.132:0/3363471765 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864042040 con 0x56149a9af810 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff85a7fc640 1 -- 172.21.15.132:0/3363471765 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864004a70 con 0x56149a9af810 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff85a7fc640 1 -- 172.21.15.132:0/3363471765 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640054a0 con 0x56149a9af810 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 -- 172.21.15.132:0/3363471765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 msgr2=0x56149a9d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff86402cfd0 tx=0x7ff864009260 comp rx=0 tx=0).stop 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 -- 172.21.15.132:0/3363471765 shutdown_connections 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9b85e0 0x56149a9b7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9af810 0x56149a9d0160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/3363471765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9aeef0 0x56149a9c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 -- 172.21.15.132:0/3363471765 >> 172.21.15.132:0/3363471765 conn(0x56149a9a9810 msgr2=0x56149a9a8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 -- 172.21.15.132:0/3363471765 shutdown_connections 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.505+0000 7ff87d7a1740 1 -- 172.21.15.132:0/3363471765 wait complete. 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 -- start start 2024-03-18T23:17:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 Processor -- start 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ab9c0 con 0x56149a9aeef0 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ace30 con 0x56149a9af810 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741ac090 con 0x56149a9b85e0 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff87d7a1740 10 client.0 _ll_register_callbacks cb 0x7ff87000f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff878878640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff878878640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40346/0 (socket says 172.21.15.132:40346) 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff878878640 1 -- 172.21.15.132:0/1338982211 learned_addr learned my addr 172.21.15.132:0/1338982211 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff879079640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.506+0000 7ff86bfff640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 -- 172.21.15.132:0/1338982211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 msgr2=0x56149a9bc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9bc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 -- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 msgr2=0x56149a9bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 -- 172.21.15.132:0/1338982211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56149aa2d790 con 0x56149a9aeef0 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff86bfff640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff879079640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff878878640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff87000eab0 tx=0x7ff87000fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68e7c630ed955a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003f060 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7ff870005250 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff864004a70 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff8700054b0 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff87d7a1740 10 client.16044 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56149aa2e9a0 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864004a70 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff870026040 con 0x56149a9aeef0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.507+0000 7ff87d7a1740 10 client.16044 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff87d7a1740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff87d7a1740 20 client.16044 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149aa2f0a0 0x56149aa314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff87d7a1740 4 client.16044 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56149a91a470 con 0x56149aa2f0a0 2024-03-18T23:17:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff86bfff640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149aa2f0a0 0x56149aa314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff853fff640 10 client.16044 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.508+0000 7ff86bfff640 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149aa2f0a0 0x56149aa314c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622d726e22b30abf server_cookie=8f773e8e7d7d5f65 in_seq=0 out_seq=0 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff853fff640 1 -- 172.21.15.132:0/1338982211 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56149a91a470 con 0x56149aa2f0a0 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff853fff640 10 client.16044 handle_command_reply: tid=0 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 client.16044 shutdown 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 msgr2=0x56149a9bbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff87000eab0 tx=0x7ff87000fa60 comp rx=0 tx=0).stop 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 shutdown_connections 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56149aa2f0a0 0x56149aa314c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56149a9b85e0 0x56149a9bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56149a9af810 0x56149a9bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff853fff640 0 client.16044 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 --2- 172.21.15.132:0/1338982211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56149a9aeef0 0x56149a9bbdd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 >> 172.21.15.132:0/1338982211 conn(0x56149a9a9810 msgr2=0x56149a9afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 shutdown_connections 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 1 -- 172.21.15.132:0/1338982211 wait complete. 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.515+0000 7ff87d7a1740 20 client.16044 trim_cache size 0 max 16384 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:17:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:17:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:17:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 msgr2=0x7ff85c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 0x7ff85c0a4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff870011440 tx=0x7ff870005f90 comp rx=0 tx=0).stop 2024-03-18T23:17:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 msgr2=0x7ff8741a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 0x7ff8741a3eb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ec50 tx=0x7ff86c00c6a0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 shutdown_connections 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff85c0a2a10 0x7ff85c0a4ed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff874106700 0x7ff8741a3eb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8741010b0 0x7ff874199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 --2- 172.21.15.132:0/2561020737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8741006f0 0x7ff874198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 >> 172.21.15.132:0/2561020737 conn(0x7ff874077850 msgr2=0x7ff874102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 shutdown_connections 2024-03-18T23:17:23.518 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:23.516+0000 7ff87aadc640 1 -- 172.21.15.132:0/2561020737 wait complete. 2024-03-18T23:17:23.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:23 smithi196 ceph-mon[67089]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:23.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:23 smithi144 ceph-mon[66781]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:23.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:23 smithi132 ceph-mon[32020]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:25.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:25 smithi196 ceph-mon[67089]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:25.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:25 smithi144 ceph-mon[66781]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:25.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:25 smithi132 ceph-mon[32020]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-03-18T23:17:26.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:26 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:26.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:26 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:26.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:26 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:17:27.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:27.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:27.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:17:27.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:27 smithi144 ceph-mon[66781]: pgmap v253: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:17:27.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:17:27.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:27 smithi132 ceph-mon[32020]: pgmap v253: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:17:27.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:17:27.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:17:27.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:17:27.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:27 smithi196 ceph-mon[67089]: pgmap v253: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:17:29.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:29 smithi196 ceph-mon[67089]: pgmap v254: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:29.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:29 smithi144 ceph-mon[66781]: pgmap v254: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:29 smithi132 ceph-mon[32020]: pgmap v254: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:31.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:31 smithi196 ceph-mon[67089]: pgmap v255: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:31 smithi144 ceph-mon[66781]: pgmap v255: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:31 smithi132 ceph-mon[32020]: pgmap v255: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:33.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:33 smithi196 ceph-mon[67089]: pgmap v256: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:33.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:33 smithi144 ceph-mon[66781]: pgmap v256: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:33.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:33 smithi132 ceph-mon[32020]: pgmap v256: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:35.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:35 smithi196 ceph-mon[67089]: pgmap v257: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:35.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:35 smithi144 ceph-mon[66781]: pgmap v257: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:35.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:35 smithi132 ceph-mon[32020]: pgmap v257: 97 pgs: 97 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 -- 172.21.15.132:0/1918863086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb081000b0 msgr2=0x7fcb08100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 --2- 172.21.15.132:0/1918863086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb081000b0 0x7fcb08100490 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009900 tx=0x7fcb0402bc30 comp rx=0 tx=0).stop 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 -- 172.21.15.132:0/1918863086 shutdown_connections 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 --2- 172.21.15.132:0/1918863086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 0x7fcb0810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 --2- 172.21.15.132:0/1918863086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb08100a60 0x7fcb08104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 --2- 172.21.15.132:0/1918863086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb081000b0 0x7fcb08100490 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 -- 172.21.15.132:0/1918863086 >> 172.21.15.132:0/1918863086 conn(0x7fcb08076c30 msgr2=0x7fcb080fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 -- 172.21.15.132:0/1918863086 shutdown_connections 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.612+0000 7fcb102f8640 1 -- 172.21.15.132:0/1918863086 wait complete. 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 Processor -- start 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 -- start start 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 0x7fcb08197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 0x7fcb08198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08110240 con 0x7fcb0819c890 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081100c0 con 0x7fcb081000b0 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb102f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081103c0 con 0x7fcb08105180 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb0e895640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 0x7fcb08197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb0e895640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40648/0 (socket says 172.21.15.132:40648) 2024-03-18T23:17:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb0e895640 1 -- 172.21.15.132:0/3842132865 learned_addr learned my addr 172.21.15.132:0/3842132865 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.613+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 0x7fcb08198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 -- 172.21.15.132:0/3842132865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 msgr2=0x7fcb08198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 0x7fcb08198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 -- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 msgr2=0x7fcb0819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 -- 172.21.15.132:0/3842132865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0819d380 con 0x7fcb081000b0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e895640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 0x7fcb08197fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb04036040 tx=0x7fcb0402cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcaff7fe640 1 -- 172.21.15.132:0/3842132865 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04041070 con 0x7fcb081000b0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0819d610 con 0x7fcb081000b0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcaff7fe640 1 -- 172.21.15.132:0/3842132865 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb04004450 con 0x7fcb081000b0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcaff7fe640 1 -- 172.21.15.132:0/3842132865 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04003d90 con 0x7fcb081000b0 2024-03-18T23:17:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.614+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb081a4de0 con 0x7fcb081000b0 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.616+0000 7fcaff7fe640 1 -- 172.21.15.132:0/3842132865 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fcb04044070 con 0x7fcb081000b0 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 Processor -- start 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 -- start start 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcaff7fe640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 0x7fcaf40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378eadb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56378ea33be0 con 0x56378ead0810 2024-03-18T23:17:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56378e9dc960 con 0x56378ead95e0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56378ea2fc80 con 0x56378eacfef0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb0d893640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb0e895640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 0x7fcaf40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb0d893640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40650/0 (socket says 172.21.15.132:40650) 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb0d893640 1 -- 172.21.15.132:0/3376281711 learned_addr learned my addr 172.21.15.132:0/3376281711 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.617+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0e895640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378eadb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcaff7fe640 1 -- 172.21.15.132:0/3842132865 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcb040f0760 con 0x7fcb081000b0 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 -- 172.21.15.132:0/3376281711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 msgr2=0x56378ead02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 -- 172.21.15.132:0/3376281711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 msgr2=0x56378eadb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378eadb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 -- 172.21.15.132:0/3376281711 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378e9d3d20 con 0x56378ead0810 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0e895640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 0x7fcaf40a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb000049d0 tx=0x7fcb000116d0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009c60 tx=0x7fcaf8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373184aa4b3454b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcafcbfe640 1 -- 172.21.15.132:0/3376281711 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8004050 con 0x56378ead0810 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcafcbfe640 1 -- 172.21.15.132:0/3376281711 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf80041f0 con 0x56378ead0810 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.618+0000 7fcafcbfe640 1 -- 172.21.15.132:0/3376281711 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8019700 con 0x56378ead0810 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3376281711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 msgr2=0x56378ead90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009c60 tx=0x7fcaf8011810 comp rx=0 tx=0).stop 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3376281711 shutdown_connections 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378eadb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead90a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3376281711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3376281711 >> 172.21.15.132:0/3376281711 conn(0x56378eaca810 msgr2=0x56378eacabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3376281711 shutdown_connections 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3376281711 wait complete. 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.619+0000 7fcb12fc2740 1 -- start start 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 Processor -- start 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 0x56378ead6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56378eadc210 con 0x56378eacfef0 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56378e9efdb0 con 0x56378ead95e0 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56378eadc390 con 0x56378ead0810 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb12fc2740 10 client.0 _ll_register_callbacks cb 0x56378e9f4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40656/0 (socket says 172.21.15.132:40656) 2024-03-18T23:17:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0d893640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 0x56378ead6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 -- 172.21.15.132:0/3709291016 learned_addr learned my addr 172.21.15.132:0/3709291016 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e895640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 -- 172.21.15.132:0/3709291016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 msgr2=0x56378ead6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 0x56378ead6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 -- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 msgr2=0x56378ead67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 -- 172.21.15.132:0/3709291016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378e9eec60 con 0x56378eacfef0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 0x56378ead6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcb0e094640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb0402cfd0 tx=0x7fcb040bc8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a482cc387eee2b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.620+0000 7fcaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0404b040 con 0x56378eacfef0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcb040ccc00 con 0x56378eacfef0 2024-03-18T23:17:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf80041f0 con 0x56378eacfef0 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcb04004c80 con 0x56378eacfef0 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 10 client.16056 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56378e748210 con 0x56378eacfef0 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf80041f0 con 0x56378eacfef0 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcb040c51e0 con 0x56378eacfef0 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 10 client.16056 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 20 client.16056 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eae9440 0x56378eaeb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 4 client.16056 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378ea41120 con 0x56378eae9440 2024-03-18T23:17:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.621+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eae9440 0x56378eaeb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcaf1ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb0d893640 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eae9440 0x56378eaeb890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ad31abb27ee715 server_cookie=60e36ee4637e6588 in_seq=0 out_seq=0 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/3709291016 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56378ea41120 con 0x56378eae9440 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcaf1ffb640 10 client.16056 handle_command_reply: tid=0 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 client.16056 shutdown 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 msgr2=0x56378ead8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb0402cfd0 tx=0x7fcb040bc8e0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 shutdown_connections 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eae9440 0x56378eaeb890 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead0810 0x56378ead6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcaf1ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.622+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/3709291016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378ead8f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.623+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 >> 172.21.15.132:0/3709291016 conn(0x56378eaca810 msgr2=0x56378eac9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.623+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 shutdown_connections 2024-03-18T23:17:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.623+0000 7fcb12fc2740 1 -- 172.21.15.132:0/3709291016 wait complete. 2024-03-18T23:17:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.623+0000 7fcb12fc2740 20 client.16056 trim_cache size 0 max 16384 2024-03-18T23:17:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 Processor -- start 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 -- start start 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 0x56378ead1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead95e0 0x56378eaf1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081afc10 con 0x56378eacfef0 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081ad4e0 con 0x56378ead0810 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.637+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081ab350 con 0x56378ead95e0 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e895640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead95e0 0x56378eaf1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40660/0 (socket says 172.21.15.132:40660) 2024-03-18T23:17:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 -- 172.21.15.132:0/4231798843 learned_addr learned my addr 172.21.15.132:0/4231798843 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0d893640 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 0x56378ead1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 -- 172.21.15.132:0/4231798843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead95e0 msgr2=0x56378eaf1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead95e0 0x56378eaf1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 -- 172.21.15.132:0/4231798843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 msgr2=0x56378ead1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 0x56378ead1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 -- 172.21.15.132:0/4231798843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378ead0640 con 0x56378eacfef0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0d893640 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 0x56378ead1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcb0e094640 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005940 tx=0x7fcb040bca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f064c45b1013a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcaf3fff640 1 -- 172.21.15.132:0/4231798843 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04004c80 con 0x56378eacfef0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcaf3fff640 1 -- 172.21.15.132:0/4231798843 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb040c1020 con 0x56378eacfef0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.638+0000 7fcaf3fff640 1 -- 172.21.15.132:0/4231798843 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040ee2d0 con 0x56378eacfef0 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- 172.21.15.132:0/4231798843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 msgr2=0x56378eae4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005940 tx=0x7fcb040bca10 comp rx=0 tx=0).stop 2024-03-18T23:17:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- 172.21.15.132:0/4231798843 shutdown_connections 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378ead95e0 0x56378eaf1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead0810 0x56378ead1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/4231798843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378eacfef0 0x56378eae4fe0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- 172.21.15.132:0/4231798843 >> 172.21.15.132:0/4231798843 conn(0x56378eaca810 msgr2=0x56378ead1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- 172.21.15.132:0/4231798843 shutdown_connections 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- 172.21.15.132:0/4231798843 wait complete. 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- start start 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 Processor -- start 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081ab9a0 con 0x56378ead0810 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081ace10 con 0x56378ead95e0 2024-03-18T23:17:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081ac070 con 0x56378eacfef0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.639+0000 7fcb12fc2740 10 client.0 _ll_register_callbacks cb 0x7fcb0402ce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0e094640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:40668/0 (socket says 172.21.15.132:40668) 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 -- 172.21.15.132:0/2544212471 learned_addr learned my addr 172.21.15.132:0/2544212471 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0e895640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 -- 172.21.15.132:0/2544212471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 msgr2=0x56378ead3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 -- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 msgr2=0x56378ead2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 -- 172.21.15.132:0/2544212471 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378e814620 con 0x56378ead0810 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb0d893640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000c00 tx=0x7fcaf8005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf41f5a749fc368 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcaf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf801f040 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcaf8009d50 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56378ea3b470 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcaf80092a0 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb12fc2740 10 client.16062 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56378eb542c0 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.640+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56378ea3b470 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcaf8018070 con 0x56378ead0810 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 20 client.16062 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eb54930 0x56378eb56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 4 client.16062 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56378e9b9770 con 0x56378eb54930 2024-03-18T23:17:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb0e094640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eb54930 0x56378eb56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcb0e094640 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eb54930 0x56378eb56d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c0d5ee6d379bda server_cookie=ef46fa86fcbb4799 in_seq=0 out_seq=0 2024-03-18T23:17:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.641+0000 7fcaf1ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcaf1ffb640 1 -- 172.21.15.132:0/2544212471 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56378e9b9770 con 0x56378eb54930 2024-03-18T23:17:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcaf1ffb640 10 client.16062 handle_command_reply: tid=0 2024-03-18T23:17:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 client.16062 shutdown 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 msgr2=0x56378ead2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000c00 tx=0x7fcaf8005cc0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 shutdown_connections 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56378eb54930 0x56378eb56d50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56378ead95e0 0x56378ead2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56378ead0810 0x56378ead2520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcaf1ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 --2- 172.21.15.132:0/2544212471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56378eacfef0 0x56378ead3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 >> 172.21.15.132:0/2544212471 conn(0x56378eaca810 msgr2=0x56378eacc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 shutdown_connections 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.648+0000 7fcb12fc2740 1 -- 172.21.15.132:0/2544212471 wait complete. 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb12fc2740 20 client.16062 trim_cache size 0 max 16384 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:17:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 msgr2=0x7fcaf40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 0x7fcaf40a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb000049d0 tx=0x7fcb000116d0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 msgr2=0x7fcb08197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 0x7fcb08197fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb04036040 tx=0x7fcb0402cb20 comp rx=0 tx=0).stop 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 shutdown_connections 2024-03-18T23:17:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fcaf40a2a10 0x7fcaf40a4ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb0819c890 0x7fcb0819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb08105180 0x7fcb08198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 --2- 172.21.15.132:0/3842132865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb081000b0 0x7fcb08197fd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 >> 172.21.15.132:0/3842132865 conn(0x7fcb08076c30 msgr2=0x7fcb080fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 shutdown_connections 2024-03-18T23:17:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:36.649+0000 7fcb102f8640 1 -- 172.21.15.132:0/3842132865 wait complete. 2024-03-18T23:17:37.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:37 smithi196 ceph-mon[67089]: pgmap v258: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:37.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:37 smithi144 ceph-mon[66781]: pgmap v258: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:37.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:37 smithi132 ceph-mon[32020]: pgmap v258: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:39 smithi196 ceph-mon[67089]: pgmap v259: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:17:39.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:39 smithi144 ceph-mon[66781]: pgmap v259: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:17:39.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:39 smithi132 ceph-mon[32020]: pgmap v259: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:17:42.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:42 smithi196 ceph-mon[67089]: pgmap v260: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:42.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:42 smithi144 ceph-mon[66781]: pgmap v260: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:42.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:42 smithi132 ceph-mon[32020]: pgmap v260: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:17:45.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:44 smithi196 ceph-mon[67089]: pgmap v261: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:45.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:44 smithi144 ceph-mon[66781]: pgmap v261: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:45.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:44 smithi132 ceph-mon[32020]: pgmap v261: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:47.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:46 smithi196 ceph-mon[67089]: pgmap v262: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:47.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:46 smithi144 ceph-mon[66781]: pgmap v262: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:47.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:46 smithi132 ceph-mon[32020]: pgmap v262: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:49.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:48 smithi196 ceph-mon[67089]: pgmap v263: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:49.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:48 smithi144 ceph-mon[66781]: pgmap v263: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:49.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:48 smithi132 ceph-mon[32020]: pgmap v263: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 -- 172.21.15.132:0/1053811708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560101310 msgr2=0x7f956010d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 --2- 172.21.15.132:0/1053811708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560101310 0x7f956010d550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9558009be0 tx=0x7f955802bef0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 -- 172.21.15.132:0/1053811708 shutdown_connections 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 --2- 172.21.15.132:0/1053811708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560101310 0x7f956010d550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 --2- 172.21.15.132:0/1053811708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9560100970 0x7f9560100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 --2- 172.21.15.132:0/1053811708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95601056a0 0x7f9560105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 -- 172.21.15.132:0/1053811708 >> 172.21.15.132:0/1053811708 conn(0x7f95600774f0 msgr2=0x7f9560077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 -- 172.21.15.132:0/1053811708 shutdown_connections 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.743+0000 7f95674f1640 1 -- 172.21.15.132:0/1053811708 wait complete. 2024-03-18T23:17:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 Processor -- start 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 -- start start 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 0x7f9560198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 0x7f9560199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110bb0 con 0x7f95601056a0 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110a30 con 0x7f9560101310 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f95674f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110d30 con 0x7f9560100970 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34894/0 (socket says 172.21.15.132:34894) 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9564a8c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 0x7f9560199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 -- 172.21.15.132:0/2325562497 learned_addr learned my addr 172.21.15.132:0/2325562497 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f956528d640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 0x7f9560198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 -- 172.21.15.132:0/2325562497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 msgr2=0x7f9560198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 0x7f9560198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 -- 172.21.15.132:0/2325562497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 msgr2=0x7f9560199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 0x7f9560199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.744+0000 7f9565a8e640 1 -- 172.21.15.132:0/2325562497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95601a4410 con 0x7f95601056a0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f956528d640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 0x7f9560198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f9564a8c640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 0x7f9560199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f9565a8e640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f95580096f0 tx=0x7f95580336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f95567fc640 1 -- 172.21.15.132:0/2325562497 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558043020 con 0x7f95601056a0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95601a45e0 con 0x7f95601056a0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f95567fc640 1 -- 172.21.15.132:0/2325562497 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95580338a0 con 0x7f95601056a0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f95567fc640 1 -- 172.21.15.132:0/2325562497 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955803e6b0 con 0x7f95601056a0 2024-03-18T23:17:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.745+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95601a4b20 con 0x7f95601056a0 2024-03-18T23:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.747+0000 7f95567fc640 1 -- 172.21.15.132:0/2325562497 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f9558046020 con 0x7f95601056a0 2024-03-18T23:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 Processor -- start 2024-03-18T23:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 -- start start 2024-03-18T23:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab26d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab2789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55adab1d0be0 con 0x55adab2765e0 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55adab179960 con 0x55adab26d810 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956a1bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adab1ccc80 con 0x55adab26cef0 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f95567fc640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 0x7f954c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f9564a8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab26d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:42476/0 (socket says 172.21.15.132:42476) 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 -- 172.21.15.132:0/925350931 learned_addr learned my addr 172.21.15.132:0/925350931 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f9564a8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab26d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:41462/0 (socket says 172.21.15.132:41462) 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f9564a8c640 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab2789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f9565a8e640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 0x7f954c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 -- 172.21.15.132:0/925350931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 msgr2=0x55adab26d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab26d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.748+0000 7f956528d640 1 -- 172.21.15.132:0/925350931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 msgr2=0x55adab2789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f956528d640 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab2789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f956528d640 1 -- 172.21.15.132:0/925350931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adab170d20 con 0x55adab26d810 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f95567fc640 1 -- 172.21.15.132:0/2325562497 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f95580f12f0 con 0x7f95601056a0 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f956528d640 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95500114c0 tx=0x7f9550011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f88c648bb02251b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f954b7fe640 1 -- 172.21.15.132:0/925350931 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550004020 con 0x55adab26d810 2024-03-18T23:17:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f9565a8e640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 0x7f954c0a4ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95580bd9f0 tx=0x7f955808f690 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f954b7fe640 1 -- 172.21.15.132:0/925350931 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95500041c0 con 0x55adab26d810 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.749+0000 7f954b7fe640 1 -- 172.21.15.132:0/925350931 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000f8f0 con 0x55adab26d810 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- 172.21.15.132:0/925350931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 msgr2=0x55adab2760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95500114c0 tx=0x7f9550011980 comp rx=0 tx=0).stop 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- 172.21.15.132:0/925350931 shutdown_connections 2024-03-18T23:17:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab2789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab2760a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 --2- 172.21.15.132:0/925350931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab26d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- 172.21.15.132:0/925350931 >> 172.21.15.132:0/925350931 conn(0x55adab267810 msgr2=0x55adab267bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- 172.21.15.132:0/925350931 shutdown_connections 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- 172.21.15.132:0/925350931 wait complete. 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 -- start start 2024-03-18T23:17:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.750+0000 7f956a1bb740 1 Processor -- start 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab275fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab273970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55adab279210 con 0x55adab2765e0 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55adab18cdb0 con 0x55adab26d810 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55adab279390 con 0x55adab26cef0 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 10 client.0 _ll_register_callbacks cb 0x55adab191e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:42486/0 (socket says 172.21.15.132:42486) 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 -- 172.21.15.132:0/1157192549 learned_addr learned my addr 172.21.15.132:0/1157192549 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f9564a8c640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab275fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f9564a8c640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab273970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 -- 172.21.15.132:0/1157192549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 msgr2=0x55adab275fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab275fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 -- 172.21.15.132:0/1157192549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 msgr2=0x55adab273970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab273970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 -- 172.21.15.132:0/1157192549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adab18bc60 con 0x55adab26d810 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956528d640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f95500096e0 tx=0x7f9550009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=977bda07aa61853e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f9548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001d040 con 0x55adab26d810 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95500041c0 con 0x55adab26d810 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.751+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f95500043f0 con 0x55adab26d810 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9550030050 con 0x55adab26d810 2024-03-18T23:17:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 10 client.25919 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55adaaee5210 con 0x55adab26d810 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95500041c0 con 0x55adab26d810 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f955002b030 con 0x55adab26d810 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 10 client.25919 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 20 client.25919 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab286450 0x55adab288840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 4 client.25919 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:17:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adab1de120 con 0x55adab286450 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.752+0000 7f9564a8c640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab286450 0x55adab288840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.753+0000 7f9548ff9640 10 client.25919 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.753+0000 7f9564a8c640 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab286450 0x55adab288840 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9e4349b8301870 server_cookie=c2bcdd946ffe2105 in_seq=0 out_seq=0 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.753+0000 7f9548ff9640 1 -- 172.21.15.132:0/1157192549 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55adab1de120 con 0x55adab286450 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.753+0000 7f9548ff9640 10 client.25919 handle_command_reply: tid=0 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 client.25919 shutdown 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 msgr2=0x55adab273360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f95500096e0 tx=0x7f9550009de0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 shutdown_connections 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab286450 0x55adab288840 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab2765e0 0x55adab273970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab273360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 --2- 172.21.15.132:0/1157192549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab26cef0 0x55adab275fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f9548ff9640 0 client.25919 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 >> 172.21.15.132:0/1157192549 conn(0x55adab267810 msgr2=0x55adab266ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 shutdown_connections 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 1 -- 172.21.15.132:0/1157192549 wait complete. 2024-03-18T23:17:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.754+0000 7f956a1bb740 20 client.25919 trim_cache size 0 max 16384 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 Processor -- start 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 -- start start 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26cef0 0x55adab281fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab28dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f95601adae0 con 0x55adab26d810 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ab3b0 con 0x55adab26cef0 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956a1bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95601a9240 con 0x55adab2765e0 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34908/0 (socket says 172.21.15.132:34908) 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 -- 172.21.15.132:0/157263350 learned_addr learned my addr 172.21.15.132:0/157263350 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f9564a8c640 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26cef0 0x55adab281fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f9564a8c640 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab28dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 -- 172.21.15.132:0/157263350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 msgr2=0x55adab28dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab28dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 -- 172.21.15.132:0/157263350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26cef0 msgr2=0x55adab281fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26cef0 0x55adab281fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 -- 172.21.15.132:0/157263350 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adab26d640 con 0x55adab26d810 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.769+0000 7f956528d640 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9550011e80 tx=0x7f9550003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ac84cd5fd54ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f954affd640 1 -- 172.21.15.132:0/157263350 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955001d040 con 0x55adab26d810 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f954affd640 1 -- 172.21.15.132:0/157263350 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955000fcd0 con 0x55adab26d810 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f954affd640 1 -- 172.21.15.132:0/157263350 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500046b0 con 0x55adab26d810 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- 172.21.15.132:0/157263350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 msgr2=0x55adab262940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9550011e80 tx=0x7f9550003bc0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- 172.21.15.132:0/157263350 shutdown_connections 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab28dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26d810 0x55adab262940 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 --2- 172.21.15.132:0/157263350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26cef0 0x55adab281fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- 172.21.15.132:0/157263350 >> 172.21.15.132:0/157263350 conn(0x55adab267810 msgr2=0x55adab266ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- 172.21.15.132:0/157263350 shutdown_connections 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- 172.21.15.132:0/157263350 wait complete. 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.770+0000 7f956a1bb740 1 -- start start 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 Processor -- start 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab26fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab270130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f95601a9890 con 0x55adab26cef0 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95601aace0 con 0x55adab26d810 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95601a9f40 con 0x55adab2765e0 2024-03-18T23:17:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956a1bb740 10 client.0 _ll_register_callbacks cb 0x7f955c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:34912/0 (socket says 172.21.15.132:34912) 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 -- 172.21.15.132:0/3725904820 learned_addr learned my addr 172.21.15.132:0/3725904820 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956528d640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab26fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab270130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 -- 172.21.15.132:0/3725904820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 msgr2=0x55adab270130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab270130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 -- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 msgr2=0x55adab26fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab26fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f9564a8c640 1 -- 172.21.15.132:0/3725904820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adaafb1620 con 0x55adab26cef0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.771+0000 7f956528d640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab26fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9564a8c640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f955c015fd0 tx=0x7f955c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd5ef062d27cdb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c003d90 con 0x55adab26cef0 2024-03-18T23:17:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f955c003f30 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955000fcd0 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f955c03e050 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 10 client.16080 fetch_fsmap learned FSMap version 24 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55adab2ec2e0 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955000fcd0 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f955c024040 con 0x55adab26cef0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 10 client.16080 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 20 client.16080 populate_metadata read hostname 'smithi132' 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab2eca60 0x55adab2eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 4 client.16080 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adab1d8470 con 0x55adab2eca60 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.772+0000 7f956528d640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab2eca60 0x55adab2eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.773+0000 7f9548ff9640 10 client.16080 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.773+0000 7f956528d640 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab2eca60 0x55adab2eee80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef5082fd6054f48 server_cookie=127eb9e131c032a9 in_seq=0 out_seq=0 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f9548ff9640 1 -- 172.21.15.132:0/3725904820 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adab1d8470 con 0x55adab2eca60 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f9548ff9640 10 client.16080 handle_command_reply: tid=0 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 client.16080 shutdown 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 msgr2=0x55adab27e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f955c015fd0 tx=0x7f955c017040 comp rx=0 tx=0).stop 2024-03-18T23:17:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 shutdown_connections 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55adab2eca60 0x55adab2eee80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adab2765e0 0x55adab270130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adab26d810 0x55adab26fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f9548ff9640 0 client.16080 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 --2- 172.21.15.132:0/3725904820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55adab26cef0 0x55adab27e7d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 >> 172.21.15.132:0/3725904820 conn(0x55adab267810 msgr2=0x55adab2694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 shutdown_connections 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 1 -- 172.21.15.132:0/3725904820 wait complete. 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.780+0000 7f956a1bb740 20 client.16080 trim_cache size 0 max 16384 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:17:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:17:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:17:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 msgr2=0x7f954c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 0x7f954c0a4ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95580bd9f0 tx=0x7f955808f690 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 msgr2=0x7f95601a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f95580096f0 tx=0x7f95580336b0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 shutdown_connections 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f954c0a2a10 0x7f954c0a4ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95601056a0 0x7f95601a3ed0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9560101310 0x7f9560199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 --2- 172.21.15.132:0/2325562497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9560100970 0x7f9560198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 >> 172.21.15.132:0/2325562497 conn(0x7f95600774f0 msgr2=0x7f95600fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 shutdown_connections 2024-03-18T23:17:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:17:49.781+0000 7f95674f1640 1 -- 172.21.15.132:0/2325562497 wait complete. 2024-03-18T23:17:51.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:50 smithi196 ceph-mon[67089]: pgmap v264: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:50 smithi144 ceph-mon[66781]: pgmap v264: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:50 smithi132 ceph-mon[32020]: pgmap v264: 97 pgs: 97 active+clean; 4.7 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:53.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:52 smithi196 ceph-mon[67089]: pgmap v265: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-18T23:17:53.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:52 smithi144 ceph-mon[66781]: pgmap v265: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-18T23:17:53.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:52 smithi132 ceph-mon[32020]: pgmap v265: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-18T23:17:55.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:54 smithi196 ceph-mon[67089]: pgmap v266: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:55.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:54 smithi144 ceph-mon[66781]: pgmap v266: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:55.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:54 smithi132 ceph-mon[32020]: pgmap v266: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:57.160 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:56 smithi144 ceph-mon[66781]: pgmap v267: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:57.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:56 smithi196 ceph-mon[67089]: pgmap v267: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:57.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:56 smithi132 ceph-mon[32020]: pgmap v267: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:17:59.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:17:58 smithi196 ceph-mon[67089]: pgmap v268: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:17:59.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:17:58 smithi144 ceph-mon[66781]: pgmap v268: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:17:59.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:17:58 smithi132 ceph-mon[32020]: pgmap v268: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:18:01.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:00 smithi196 ceph-mon[67089]: pgmap v269: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:01.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:00 smithi144 ceph-mon[66781]: pgmap v269: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:01.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:00 smithi132 ceph-mon[32020]: pgmap v269: 97 pgs: 97 active+clean; 4.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 -- 172.21.15.132:0/3432650623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8008068ea0 msgr2=0x7f8008069320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 --2- 172.21.15.132:0/3432650623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8008068ea0 0x7f8008069320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009900 tx=0x7f7ff802bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 -- 172.21.15.132:0/3432650623 shutdown_connections 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 --2- 172.21.15.132:0/3432650623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f800810e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 --2- 172.21.15.132:0/3432650623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8008068ea0 0x7f8008069320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 --2- 172.21.15.132:0/3432650623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80081077a0 0x7f8008107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 -- 172.21.15.132:0/3432650623 >> 172.21.15.132:0/3432650623 conn(0x7f80080774b0 msgr2=0x7f80080778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 -- 172.21.15.132:0/3432650623 shutdown_connections 2024-03-18T23:18:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.876+0000 7f801060c640 1 -- 172.21.15.132:0/3432650623 wait complete. 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 Processor -- start 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 -- start start 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 0x7f8008198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 0x7f800819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111bb0 con 0x7f8008069860 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111a30 con 0x7f80081077a0 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f801060c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111d30 con 0x7f8008068ea0 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800dba7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 0x7f8008198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800dba7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54680/0 (socket says 172.21.15.132:54680) 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 0x7f8008198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:60892/0 (socket says 172.21.15.132:60892) 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800dba7640 1 -- 172.21.15.132:0/1376143403 learned_addr learned my addr 172.21.15.132:0/1376143403 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.877+0000 7f800eba9640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 0x7f800819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 -- 172.21.15.132:0/1376143403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 msgr2=0x7f8008198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 0x7f8008198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 -- 172.21.15.132:0/1376143403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 msgr2=0x7f800819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 0x7f800819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 -- 172.21.15.132:0/1376143403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800819cff0 con 0x7f8008069860 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800eba9640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 0x7f800819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f800dba7640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005940 tx=0x7f7ff8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f7fff7fe640 1 -- 172.21.15.132:0/1376143403 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8041070 con 0x7f8008069860 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f7fff7fe640 1 -- 172.21.15.132:0/1376143403 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff80054e0 con 0x7f8008069860 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800819d280 con 0x7f8008069860 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f7fff7fe640 1 -- 172.21.15.132:0/1376143403 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8003d00 con 0x7f8008069860 2024-03-18T23:18:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.878+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a4b70 con 0x7f8008069860 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.880+0000 7f7fff7fe640 1 -- 172.21.15.132:0/1376143403 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f7ff8044070 con 0x7f8008069860 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.881+0000 7f80131f1740 1 Processor -- start 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.881+0000 7f80131f1740 1 -- start start 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.881+0000 7f7fff7fe640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 0x7f7ff40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.881+0000 7f80131f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.881+0000 7f80131f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db53340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f80131f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db53369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f80131f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561db528ebe0 con 0x561db532b810 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f80131f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561db5237960 con 0x561db532aef0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f80131f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561db528ac80 con 0x561db53345e0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800eba9640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 0x7f7ff40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:57950/0 (socket says 172.21.15.132:57950) 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800dba7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db53340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 -- 172.21.15.132:0/3714186853 learned_addr learned my addr 172.21.15.132:0/3714186853 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800eba9640 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db53369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f7fff7fe640 1 -- 172.21.15.132:0/1376143403 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f7ff80f09b0 con 0x7f8008069860 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 -- 172.21.15.132:0/3714186853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 msgr2=0x561db53369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db53369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 -- 172.21.15.132:0/3714186853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 msgr2=0x561db53340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db53340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 -- 172.21.15.132:0/3714186853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db522ed20 con 0x561db532aef0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800eba9640 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db53369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800e3a8640 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8004009ca0 tx=0x7f8004011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2945b8c2a04b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f800eba9640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 0x7f7ff40a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x561db5326c30 tx=0x7f8000011440 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.882+0000 7f7ffcbfe640 1 -- 172.21.15.132:0/3714186853 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800401e070 con 0x561db532aef0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f7ffcbfe640 1 -- 172.21.15.132:0/3714186853 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8004019b10 con 0x561db532aef0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f7ffcbfe640 1 -- 172.21.15.132:0/3714186853 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400eae0 con 0x561db532aef0 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 -- 172.21.15.132:0/3714186853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 msgr2=0x561db532b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8004009ca0 tx=0x7f8004011810 comp rx=0 tx=0).stop 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 -- 172.21.15.132:0/3714186853 shutdown_connections 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db53369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db53340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 --2- 172.21.15.132:0/3714186853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532b2d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 -- 172.21.15.132:0/3714186853 >> 172.21.15.132:0/3714186853 conn(0x561db5325810 msgr2=0x561db5325bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 -- 172.21.15.132:0/3714186853 shutdown_connections 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.883+0000 7f80131f1740 1 -- 172.21.15.132:0/3714186853 wait complete. 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 -- start start 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 Processor -- start 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532aef0 0x561db5334030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db5331b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561db5337210 con 0x561db53345e0 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561db524adb0 con 0x561db532b810 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561db5337390 con 0x561db532aef0 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f80131f1740 10 client.0 _ll_register_callbacks cb 0x561db524fe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800eba9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db5331b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:57964/0 (socket says 172.21.15.132:57964) 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532aef0 0x561db5334030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 -- 172.21.15.132:0/1473941839 learned_addr learned my addr 172.21.15.132:0/1473941839 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 -- 172.21.15.132:0/1473941839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532aef0 msgr2=0x561db5334030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532aef0 0x561db5334030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 -- 172.21.15.132:0/1473941839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 msgr2=0x561db5331b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db5331b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800dba7640 1 -- 172.21.15.132:0/1473941839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db5249c60 con 0x561db532b810 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.884+0000 7f800eba9640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db5331b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f800dba7640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80bd140 tx=0x7f7ff8003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699fb51e41da6942 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80dbe60 con 0x561db532b810 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f7ff80de3f0 con 0x561db532b810 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8004019b10 con 0x561db532b810 2024-03-18T23:18:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7ff804b040 con 0x561db532b810 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 10 client.25937 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561db4fa3210 con 0x561db532b810 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8004019b10 con 0x561db532b810 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7ff80b9820 con 0x561db532b810 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 10 client.25937 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 20 client.25937 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53444b0 0x561db53468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 4 client.25937 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.885+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db529c120 con 0x561db53444b0 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.886+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53444b0 0x561db53468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.886+0000 7f7ff1ffb640 10 client.25937 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.886+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53444b0 0x561db53468f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff858cf033bda742 server_cookie=907b27b148ebad8e in_seq=0 out_seq=0 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1473941839 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561db529c120 con 0x561db53444b0 2024-03-18T23:18:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f7ff1ffb640 10 client.25937 handle_command_reply: tid=0 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 client.25937 shutdown 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 msgr2=0x561db53313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80bd140 tx=0x7f7ff8003760 comp rx=0 tx=0).stop 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 shutdown_connections 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53444b0 0x561db53468f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db5331b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532b810 0x561db53313c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 --2- 172.21.15.132:0/1473941839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532aef0 0x561db5334030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f7ff1ffb640 0 client.25937 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 >> 172.21.15.132:0/1473941839 conn(0x561db5325810 msgr2=0x561db5324ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 shutdown_connections 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 1 -- 172.21.15.132:0/1473941839 wait complete. 2024-03-18T23:18:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.887+0000 7f80131f1740 20 client.25937 trim_cache size 0 max 16384 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 Processor -- start 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 -- start start 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db533ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532b810 0x561db5325bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f80081afbc0 con 0x561db53345e0 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ad490 con 0x561db532aef0 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f80131f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ab300 con 0x561db532b810 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54690/0 (socket says 172.21.15.132:54690) 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 -- 172.21.15.132:0/1236650551 learned_addr learned my addr 172.21.15.132:0/1236650551 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db533ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800dba7640 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532b810 0x561db5325bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 -- 172.21.15.132:0/1236650551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532b810 msgr2=0x561db5325bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532b810 0x561db5325bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 -- 172.21.15.132:0/1236650551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 msgr2=0x561db533ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db533ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.902+0000 7f800eba9640 1 -- 172.21.15.132:0/1236650551 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db5336d40 con 0x561db53345e0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f800eba9640 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f800000fdc0 tx=0x7f8000009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bf033c67e1feab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f7ff3fff640 1 -- 172.21.15.132:0/1236650551 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800002d070 con 0x561db53345e0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f7ff3fff640 1 -- 172.21.15.132:0/1236650551 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8000003c80 con 0x561db53345e0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f7ff3fff640 1 -- 172.21.15.132:0/1236650551 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000028a80 con 0x561db53345e0 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 -- 172.21.15.132:0/1236650551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 msgr2=0x561db534c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f800000fdc0 tx=0x7f8000009500 comp rx=0 tx=0).stop 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 -- 172.21.15.132:0/1236650551 shutdown_connections 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db53345e0 0x561db534c200 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db532b810 0x561db5325bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 --2- 172.21.15.132:0/1236650551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db533ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.903+0000 7f80131f1740 1 -- 172.21.15.132:0/1236650551 >> 172.21.15.132:0/1236650551 conn(0x561db5325810 msgr2=0x561db5324ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- 172.21.15.132:0/1236650551 shutdown_connections 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- 172.21.15.132:0/1236650551 wait complete. 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- start start 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 Processor -- start 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db532e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ab950 con 0x561db532b810 2024-03-18T23:18:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80081acdc0 con 0x561db532aef0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80081ac020 con 0x561db53345e0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f80131f1740 10 client.0 _ll_register_callbacks cb 0x7f7ff80b9450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f800dba7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f800e3a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f800dba7640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54696/0 (socket says 172.21.15.132:54696) 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f800eba9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db532e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.904+0000 7f800dba7640 1 -- 172.21.15.132:0/1522562307 learned_addr learned my addr 172.21.15.132:0/1522562307 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 -- 172.21.15.132:0/1522562307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 msgr2=0x561db532e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db532e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 -- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 msgr2=0x561db532df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 -- 172.21.15.132:0/1522562307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db506f620 con 0x561db532b810 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f800dba7640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b9610 tx=0x7f7ff80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a6d8e0b7383b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80bccf0 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f7ff80c5620 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8000003c80 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7ff802ccd0 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 10 client.16098 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561db506f620 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8000003c80 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7ff80ee2d0 con 0x561db532b810 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 10 client.16098 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 20 client.16098 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53aadb0 0x561db53ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 4 client.16098 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.905+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561db5296470 con 0x561db53aadb0 2024-03-18T23:18:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.906+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53aadb0 0x561db53ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.906+0000 7f800e3a8640 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53aadb0 0x561db53ad1d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ea3ef67ff0a99c server_cookie=628b1847a9a3487e in_seq=0 out_seq=0 2024-03-18T23:18:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.906+0000 7f7ff1ffb640 10 client.16098 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f7ff1ffb640 1 -- 172.21.15.132:0/1522562307 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561db5296470 con 0x561db53aadb0 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f7ff1ffb640 10 client.16098 handle_command_reply: tid=0 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 client.16098 shutdown 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 msgr2=0x561db532e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b9610 tx=0x7f7ff80bd040 comp rx=0 tx=0).stop 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 shutdown_connections 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561db53aadb0 0x561db53ad1d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561db53345e0 0x561db532e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561db532b810 0x561db532e440 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f7ff1ffb640 0 client.16098 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.913+0000 7f80131f1740 1 --2- 172.21.15.132:0/1522562307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561db532aef0 0x561db532df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 >> 172.21.15.132:0/1522562307 conn(0x561db5325810 msgr2=0x561db53274f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 shutdown_connections 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f80131f1740 1 -- 172.21.15.132:0/1522562307 wait complete. 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f80131f1740 20 client.16098 trim_cache size 0 max 16384 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:18:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 msgr2=0x7f7ff40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 0x7f7ff40a4ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x561db5326c30 tx=0x7f8000011440 comp rx=0 tx=0).stop 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 msgr2=0x7f8008198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005940 tx=0x7f7ff8004270 comp rx=0 tx=0).stop 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 shutdown_connections 2024-03-18T23:18:02.916 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f7ff40a2a10 0x7f7ff40a4ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80081077a0 0x7f800819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8008069860 0x7f8008198590 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 --2- 172.21.15.132:0/1376143403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8008068ea0 0x7f8008198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 >> 172.21.15.132:0/1376143403 conn(0x7f80080774b0 msgr2=0x7f80080fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 shutdown_connections 2024-03-18T23:18:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:02.914+0000 7f801060c640 1 -- 172.21.15.132:0/1376143403 wait complete. 2024-03-18T23:18:03.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:02 smithi196 ceph-mon[67089]: pgmap v270: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:18:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:02 smithi144 ceph-mon[66781]: pgmap v270: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:18:03.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:02 smithi132 ceph-mon[32020]: pgmap v270: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:18:05.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:04 smithi196 ceph-mon[67089]: pgmap v271: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-18T23:18:05.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:04 smithi144 ceph-mon[66781]: pgmap v271: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-18T23:18:05.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:04 smithi132 ceph-mon[32020]: pgmap v271: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-18T23:18:06.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:05 smithi196 ceph-mon[67089]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:18:06.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:05 smithi144 ceph-mon[66781]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:18:06.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:05 smithi132 ceph-mon[32020]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T23:18:07.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:06 smithi196 ceph-mon[67089]: pgmap v273: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 31 op/s 2024-03-18T23:18:07.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:06 smithi144 ceph-mon[66781]: pgmap v273: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 31 op/s 2024-03-18T23:18:07.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:06 smithi132 ceph-mon[32020]: pgmap v273: 97 pgs: 97 active+clean; 4.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 31 op/s 2024-03-18T23:18:09.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:08 smithi196 ceph-mon[67089]: pgmap v274: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:18:09.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:08 smithi196 ceph-mon[67089]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:18:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:08 smithi144 ceph-mon[66781]: pgmap v274: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:18:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:08 smithi144 ceph-mon[66781]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:18:09.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:08 smithi132 ceph-mon[32020]: pgmap v274: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:18:09.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:08 smithi132 ceph-mon[32020]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T23:18:11.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:10 smithi196 ceph-mon[67089]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:11.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:10 smithi144 ceph-mon[66781]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:11.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:10 smithi132 ceph-mon[32020]: pgmap v276: 97 pgs: 97 active+clean; 4.9 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:13.167 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:12 smithi132 ceph-mon[32020]: pgmap v277: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:18:13.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:12 smithi196 ceph-mon[67089]: pgmap v277: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:18:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:12 smithi144 ceph-mon[66781]: pgmap v277: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:18:15.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:14 smithi196 ceph-mon[67089]: pgmap v278: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:18:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:14 smithi144 ceph-mon[66781]: pgmap v278: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:18:15.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:14 smithi132 ceph-mon[32020]: pgmap v278: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:18:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 -- 172.21.15.132:0/533033812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52701077f0 msgr2=0x7f5270107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 --2- 172.21.15.132:0/533033812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52701077f0 0x7f5270107bd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5260009900 tx=0x7f526002bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 -- 172.21.15.132:0/533033812 shutdown_connections 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 --2- 172.21.15.132:0/533033812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f527010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 --2- 172.21.15.132:0/533033812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f527006a060 0x7f527006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 --2- 172.21.15.132:0/533033812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52701077f0 0x7f5270107bd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 -- 172.21.15.132:0/533033812 >> 172.21.15.132:0/533033812 conn(0x7f5270077440 msgr2=0x7f5270077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 -- 172.21.15.132:0/533033812 shutdown_connections 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.009+0000 7f5274d1b640 1 -- 172.21.15.132:0/533033812 wait complete. 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 Processor -- start 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 -- start start 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 0x7f5270197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 0x7f527019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111be0 con 0x7f527006a060 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111a60 con 0x7f527006aa00 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f5274d1b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5270111d60 con 0x7f52701077f0 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 0x7f5270197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:56770/0 (socket says 172.21.15.132:56770) 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 0x7f527019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 -- 172.21.15.132:0/1457351687 learned_addr learned my addr 172.21.15.132:0/1457351687 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526e59c640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 0x7f5270197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46694/0 (socket says 172.21.15.132:46694) 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 -- 172.21.15.132:0/1457351687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 msgr2=0x7f527019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 0x7f527019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 -- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 msgr2=0x7f5270197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 0x7f5270197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526dd9b640 1 -- 172.21.15.132:0/1457351687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527019cf90 con 0x7f527006aa00 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.010+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 0x7f527019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f525f7fe640 1 -- 172.21.15.132:0/1457351687 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526400cd80 con 0x7f527006aa00 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f525f7fe640 1 -- 172.21.15.132:0/1457351687 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5264005500 con 0x7f527006aa00 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527019d280 con 0x7f527006aa00 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f525f7fe640 1 -- 172.21.15.132:0/1457351687 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264010690 con 0x7f527006aa00 2024-03-18T23:18:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.011+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52701a4b10 con 0x7f527006aa00 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.013+0000 7f525f7fe640 1 -- 172.21.15.132:0/1457351687 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f52640046c0 con 0x7f527006aa00 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.013+0000 7f52779e3740 1 Processor -- start 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.013+0000 7f52779e3740 1 -- start start 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f525f7fe640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 0x7f52580a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d2b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559914c8ebe0 con 0x559914d2b810 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559914c37960 con 0x559914d2aef0 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f52779e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559914c8ac80 con 0x559914d345e0 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 0x7f52580a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46710/0 (socket says 172.21.15.132:46710) 2024-03-18T23:18:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d2b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 -- 172.21.15.132:0/4180683007 learned_addr learned my addr 172.21.15.132:0/4180683007 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526ed9d640 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f525f7fe640 1 -- 172.21.15.132:0/1457351687 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f52640cf230 con 0x7f527006aa00 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 -- 172.21.15.132:0/4180683007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 msgr2=0x559914d369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 -- 172.21.15.132:0/4180683007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 msgr2=0x559914d2b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d2b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 -- 172.21.15.132:0/4180683007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559914c2ed20 con 0x559914d2b810 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526e59c640 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d2b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526dd9b640 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f526400cfd0 tx=0x7f5264005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c5a6359731d53d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 0x7f52580a4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5268009fd0 tx=0x7f52680116b0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.014+0000 7f525cff9640 1 -- 172.21.15.132:0/4180683007 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526409b190 con 0x559914d2b810 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f525cff9640 1 -- 172.21.15.132:0/4180683007 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526409bf30 con 0x559914d2b810 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f525cff9640 1 -- 172.21.15.132:0/4180683007 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640b48f0 con 0x559914d2b810 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 -- 172.21.15.132:0/4180683007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 msgr2=0x559914d340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f526400cfd0 tx=0x7f5264005740 comp rx=0 tx=0).stop 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 -- 172.21.15.132:0/4180683007 shutdown_connections 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d340a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 --2- 172.21.15.132:0/4180683007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d2b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 -- 172.21.15.132:0/4180683007 >> 172.21.15.132:0/4180683007 conn(0x559914d25810 msgr2=0x559914d25bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 -- 172.21.15.132:0/4180683007 shutdown_connections 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.015+0000 7f52779e3740 1 -- 172.21.15.132:0/4180683007 wait complete. 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 -- start start 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 Processor -- start 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x559914d37210 con 0x559914d2b810 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559914c4adb0 con 0x559914d2aef0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559914d37390 con 0x559914d345e0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f52779e3740 10 client.0 _ll_register_callbacks cb 0x559914c4fe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46726/0 (socket says 172.21.15.132:46726) 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 -- 172.21.15.132:0/1593827297 learned_addr learned my addr 172.21.15.132:0/1593827297 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 -- 172.21.15.132:0/1593827297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 msgr2=0x559914d31970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d31970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 -- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 msgr2=0x559914d33fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d33fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526dd9b640 1 -- 172.21.15.132:0/1593827297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559914c49c60 con 0x559914d2b810 2024-03-18T23:18:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.016+0000 7f526e59c640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d33fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f52640a3aa0 tx=0x7f52640a4150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f7def626c22285 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640b4ad0 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f52640b32e0 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526409bf30 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52640b2f60 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 10 client.16110 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5599149a3210 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526409bf30 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f526409ba90 con 0x559914d2b810 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 10 client.16110 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 20 client.16110 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914d44380 0x559914d46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 4 client.16110 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559914c9c120 con 0x559914d44380 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.017+0000 7f526e59c640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914d44380 0x559914d46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.018+0000 7f526e59c640 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914d44380 0x559914d46770 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ad6e20e57d6fd2 server_cookie=239508df8c9f96e7 in_seq=0 out_seq=0 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.018+0000 7f5255ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.018+0000 7f5255ffb640 1 -- 172.21.15.132:0/1593827297 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559914c9c120 con 0x559914d44380 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.018+0000 7f5255ffb640 10 client.16110 handle_command_reply: tid=0 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.018+0000 7f52779e3740 1 client.16110 shutdown 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 msgr2=0x559914d31360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f52640a3aa0 tx=0x7f52640a4150 comp rx=0 tx=0).stop 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 shutdown_connections 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914d44380 0x559914d46770 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d345e0 0x559914d31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f5255ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d2b810 0x559914d31360 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 --2- 172.21.15.132:0/1593827297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d33fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 >> 172.21.15.132:0/1593827297 conn(0x559914d25810 msgr2=0x559914d24ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 shutdown_connections 2024-03-18T23:18:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 1 -- 172.21.15.132:0/1593827297 wait complete. 2024-03-18T23:18:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.019+0000 7f52779e3740 20 client.16110 trim_cache size 0 max 16384 2024-03-18T23:18:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.033+0000 7f52779e3740 1 Processor -- start 2024-03-18T23:18:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.033+0000 7f52779e3740 1 -- start start 2024-03-18T23:18:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 0x559914d49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2b810 0x559914d33a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f52701afc30 con 0x559914d345e0 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ad500 con 0x559914d2b810 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f52779e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ab370 con 0x559914d2aef0 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 0x559914d49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46738/0 (socket says 172.21.15.132:46738) 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 -- 172.21.15.132:0/1539468105 learned_addr learned my addr 172.21.15.132:0/1539468105 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526dd9b640 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2b810 0x559914d33a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 -- 172.21.15.132:0/1539468105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 msgr2=0x559914d49790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 0x559914d49790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 -- 172.21.15.132:0/1539468105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2b810 msgr2=0x559914d33a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2b810 0x559914d33a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 -- 172.21.15.132:0/1539468105 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559914a6f620 con 0x559914d345e0 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526e59c640 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 0x559914d49790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f526ed9d640 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5268005fd0 tx=0x7f5268021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265b1eeeb72ba715 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f5257fff640 1 -- 172.21.15.132:0/1539468105 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268030070 con 0x559914d345e0 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f5257fff640 1 -- 172.21.15.132:0/1539468105 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5268009e00 con 0x559914d345e0 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.034+0000 7f5257fff640 1 -- 172.21.15.132:0/1539468105 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268020650 con 0x559914d345e0 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- 172.21.15.132:0/1539468105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 msgr2=0x559914d33fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5268005fd0 tx=0x7f5268021000 comp rx=0 tx=0).stop 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- 172.21.15.132:0/1539468105 shutdown_connections 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d33fd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2b810 0x559914d33a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- 172.21.15.132:0/1539468105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2aef0 0x559914d49790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- 172.21.15.132:0/1539468105 >> 172.21.15.132:0/1539468105 conn(0x559914d25810 msgr2=0x559914d24ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- 172.21.15.132:0/1539468105 shutdown_connections 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- 172.21.15.132:0/1539468105 wait complete. 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- start start 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 Processor -- start 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d49860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2b810 0x559914d46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ab9c0 con 0x559914d345e0 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ace30 con 0x559914d2aef0 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52701ac090 con 0x559914d2b810 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.035+0000 7f52779e3740 10 client.0 _ll_register_callbacks cb 0x7f5264005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46746/0 (socket says 172.21.15.132:46746) 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 -- 172.21.15.132:0/4099605843 learned_addr learned my addr 172.21.15.132:0/4099605843 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526e59c640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d49860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526dd9b640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2b810 0x559914d46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 -- 172.21.15.132:0/4099605843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2b810 msgr2=0x559914d46d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2b810 0x559914d46d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 -- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 msgr2=0x559914d49860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d49860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 -- 172.21.15.132:0/4099605843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559914d47850 con 0x559914d345e0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f526ed9d640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f526802e040 tx=0x7f5268005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc677e86abd728f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f5255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526803b060 con 0x559914d345e0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f526801a070 con 0x559914d345e0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5268009e00 con 0x559914d345e0 2024-03-18T23:18:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5268030030 con 0x559914d345e0 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f52779e3740 10 client.16116 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559914daa0c0 con 0x559914d345e0 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.036+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5268009e00 con 0x559914d345e0 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f526800e430 con 0x559914d345e0 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 10 client.16116 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 20 client.16116 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914daa780 0x559914dacb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 4 client.16116 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559914c96470 con 0x559914daa780 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f526e59c640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914daa780 0x559914dacb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f5255ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.037+0000 7f526e59c640 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914daa780 0x559914dacb70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e90d171f10a14d26 server_cookie=2f47c4876454b24 in_seq=0 out_seq=0 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f5255ffb640 1 -- 172.21.15.132:0/4099605843 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559914c96470 con 0x559914daa780 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f5255ffb640 10 client.16116 handle_command_reply: tid=0 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 client.16116 shutdown 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 msgr2=0x559914d47290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f526802e040 tx=0x7f5268005bd0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 shutdown_connections 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x559914daa780 0x559914dacb70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559914d345e0 0x559914d47290 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f5255ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559914d2b810 0x559914d46d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 --2- 172.21.15.132:0/4099605843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559914d2aef0 0x559914d49860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 >> 172.21.15.132:0/4099605843 conn(0x559914d25810 msgr2=0x559914d2bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 shutdown_connections 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 1 -- 172.21.15.132:0/4099605843 wait complete. 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.045+0000 7f52779e3740 20 client.16116 trim_cache size 0 max 16384 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:18:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:18:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 msgr2=0x7f52580a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 0x7f52580a4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5268009fd0 tx=0x7f52680116b0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 msgr2=0x7f52701984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f526400e9f0 tx=0x7f526400edf0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 shutdown_connections 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f52580a2b10 0x7f52580a4fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52701077f0 0x7f527019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f527006aa00 0x7f52701984e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 --2- 172.21.15.132:0/1457351687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f527006a060 0x7f5270197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 >> 172.21.15.132:0/1457351687 conn(0x7f5270077440 msgr2=0x7f52700ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 shutdown_connections 2024-03-18T23:18:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:16.046+0000 7f5274d1b640 1 -- 172.21.15.132:0/1457351687 wait complete. 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:437/600: open fd = 3 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:438/600: open fd = 3 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.189 INFO:tasks.workunit.client.0.smithi132.stdout:439/600: open fd = 3 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:440/600: open fd = 3 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:441/600: open fd = 3 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:442/600: open fd = 3 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.190 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:443/600: open fd = 3 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:444/600: open fd = 3 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:445/600: open fd = 3 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:446/600: open fd = 3 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.191 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:447/600: open fd = 3 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:448/600: open fd = 3 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:449/600: open fd = 3 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:450/600: open fd = 3 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.192 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:451/600: open fd = 3 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:452/600: open fd = 3 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:453/600: open fd = 3 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:454/600: open fd = 3 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.193 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:455/600: open fd = 3 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:456/600: open fd = 3 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:457/600: open fd = 3 2024-03-18T23:18:17.194 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:458/600: open fd = 3 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:459/600: open fd = 3 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:460/600: open fd = 3 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.195 INFO:tasks.workunit.client.0.smithi132.stdout:461/600: open fd = 3 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:462/600: open fd = 3 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:463/600: open fd = 3 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:464/600: open fd = 3 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.196 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:465/600: open fd = 3 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:466/600: open fd = 3 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:467/600: open fd = 3 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.197 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:468/600: open fd = 3 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:469/600: open fd = 3 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:470/600: open fd = 3 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.198 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:471/600: open fd = 3 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:472/600: open fd = 3 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:473/600: open fd = 3 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:474/600: open fd = 3 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.199 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:475/600: open fd = 3 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:476/600: open fd = 3 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:477/600: open fd = 3 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:478/600: open fd = 3 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.200 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:479/600: open fd = 3 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:480/600: open fd = 3 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:481/600: open fd = 3 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.201 INFO:tasks.workunit.client.0.smithi132.stdout:482/600: open fd = 3 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:483/600: open fd = 3 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:484/600: open fd = 3 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:485/600: open fd = 3 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.202 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:486/600: open fd = 3 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:487/600: open fd = 3 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:488/600: open fd = 3 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.203 INFO:tasks.workunit.client.0.smithi132.stdout:489/600: open fd = 3 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:490/600: open fd = 3 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:491/600: open fd = 3 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.204 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:492/600: open fd = 3 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:493/600: open fd = 3 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:494/600: open fd = 3 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.205 INFO:tasks.workunit.client.0.smithi132.stdout:495/600: open fd = 3 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:496/600: open fd = 3 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:497/600: open fd = 3 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:498/600: open fd = 3 2024-03-18T23:18:17.206 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.207 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:18:17.207 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:18:17.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:16 smithi196 ceph-mon[67089]: pgmap v279: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:18:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:16 smithi144 ceph-mon[66781]: pgmap v279: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:18:17.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:16 smithi132 ceph-mon[32020]: pgmap v279: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:18:19.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:18 smithi196 ceph-mon[67089]: pgmap v280: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:19.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:18 smithi144 ceph-mon[66781]: pgmap v280: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:19.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:18 smithi132 ceph-mon[32020]: pgmap v280: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:21.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:20 smithi196 ceph-mon[67089]: pgmap v281: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:21.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:20 smithi144 ceph-mon[66781]: pgmap v281: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:21.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:20 smithi132 ceph-mon[32020]: pgmap v281: 97 pgs: 97 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:18:23.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:22 smithi196 ceph-mon[67089]: pgmap v282: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:23.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:22 smithi144 ceph-mon[66781]: pgmap v282: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:22 smithi132 ceph-mon[32020]: pgmap v282: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:25.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:24 smithi196 ceph-mon[67089]: pgmap v283: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:24 smithi144 ceph-mon[66781]: pgmap v283: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:25.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:24 smithi132 ceph-mon[32020]: pgmap v283: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:27.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:26 smithi196 ceph-mon[67089]: pgmap v284: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:27.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:26 smithi144 ceph-mon[66781]: pgmap v284: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:27.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:26 smithi132 ceph-mon[32020]: pgmap v284: 97 pgs: 97 active+clean; 5.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:28.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:28.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:28.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:28.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:27 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:18:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:27 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:18:28.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:18:28.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:18:28.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:18:28.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:27 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 -- 172.21.15.132:0/1202583459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101330 msgr2=0x7f539c10d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1202583459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101330 0x7f539c10d550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5394009cd0 tx=0x7f539402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 -- 172.21.15.132:0/1202583459 shutdown_connections 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1202583459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c101330 0x7f539c10d550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1202583459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c100970 0x7f539c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1202583459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c1056c0 0x7f539c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 -- 172.21.15.132:0/1202583459 >> 172.21.15.132:0/1202583459 conn(0x7f539c077570 msgr2=0x7f539c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 -- 172.21.15.132:0/1202583459 shutdown_connections 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.141+0000 7f53a25b2640 1 -- 172.21.15.132:0/1202583459 wait complete. 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 Processor -- start 2024-03-18T23:18:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 -- start start 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 0x7f539c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 0x7f539c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f539c110be0 con 0x7f539c101330 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f539c110a60 con 0x7f539c100970 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a25b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f539c110d60 con 0x7f539c1056c0 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 0x7f539c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f539bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f539bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:52578/0 (socket says 172.21.15.132:52578) 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f539bfff640 1 -- 172.21.15.132:0/1487033624 learned_addr learned my addr 172.21.15.132:0/1487033624 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 0x7f539c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.142+0000 7f539bfff640 1 -- 172.21.15.132:0/1487033624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 msgr2=0x7f539c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539bfff640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 0x7f539c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539bfff640 1 -- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 msgr2=0x7f539c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539bfff640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 0x7f539c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539bfff640 1 -- 172.21.15.132:0/1487033624 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c19d1c0 con 0x7f539c100970 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 0x7f539c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 0x7f539c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f539bfff640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ea60 tx=0x7f538c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53997fa640 1 -- 172.21.15.132:0/1487033624 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c00cde0 con 0x7f539c100970 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53997fa640 1 -- 172.21.15.132:0/1487033624 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c0035f0 con 0x7f539c100970 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c19d4b0 con 0x7f539c100970 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53997fa640 1 -- 172.21.15.132:0/1487033624 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c010670 con 0x7f539c100970 2024-03-18T23:18:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.143+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a4d40 con 0x7f539c100970 2024-03-18T23:18:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.145+0000 7f53997fa640 1 -- 172.21.15.132:0/1487033624 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f538c0037a0 con 0x7f539c100970 2024-03-18T23:18:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.145+0000 7f53997fa640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 0x7f53880a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.145+0000 7f53a51a0740 1 Processor -- start 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 -- start start 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 0x7f53880a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53997fa640 1 -- 172.21.15.132:0/1487033624 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f538c0cf0e0 con 0x7f539c100970 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56362be62be0 con 0x56362bf0c9f0 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56362be0b960 con 0x56362befea10 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a51a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56362be5ec80 con 0x56362beff330 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a0b4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:52590/0 (socket says 172.21.15.132:52590) 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a0b4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54046/0 (socket says 172.21.15.132:54046) 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 -- 172.21.15.132:0/1069597601 learned_addr learned my addr 172.21.15.132:0/1069597601 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:33858/0 (socket says 172.21.15.132:33858) 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 0x7f53880a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f539c077bb0 tx=0x7f5384011860 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 -- 172.21.15.132:0/1069597601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 msgr2=0x56362bf0c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 -- 172.21.15.132:0/1069597601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 msgr2=0x56362bf0ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 -- 172.21.15.132:0/1069597601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362be02d20 con 0x56362befea10 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.146+0000 7f539bfff640 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f538c0a39b0 tx=0x7f538c09b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2ad98d45b7c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53927fc640 1 -- 172.21.15.132:0/1069597601 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b4070 con 0x56362befea10 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53927fc640 1 -- 172.21.15.132:0/1069597601 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c09f890 con 0x56362befea10 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53927fc640 1 -- 172.21.15.132:0/1069597601 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c097db0 con 0x56362befea10 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 -- 172.21.15.132:0/1069597601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 msgr2=0x56362befedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f538c0a39b0 tx=0x7f538c09b0a0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 -- 172.21.15.132:0/1069597601 shutdown_connections 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf0ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362beff330 0x56362bf0c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1069597601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362befea10 0x56362befedf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 -- 172.21.15.132:0/1069597601 >> 172.21.15.132:0/1069597601 conn(0x56362bef9810 msgr2=0x56362bef9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 -- 172.21.15.132:0/1069597601 shutdown_connections 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.147+0000 7f53a51a0740 1 -- 172.21.15.132:0/1069597601 wait complete. 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 -- start start 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 Processor -- start 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56362bf0f5b0 con 0x56362bf0c9f0 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56362be1edb0 con 0x56362beff330 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56362bf0f730 con 0x56362befea10 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a51a0740 10 client.0 _ll_register_callbacks cb 0x56362be23e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a0b4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f539bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a0b4f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54048/0 (socket says 172.21.15.132:54048) 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f539bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:33860/0 (socket says 172.21.15.132:33860) 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f53a0b4f640 1 -- 172.21.15.132:0/3031020591 learned_addr learned my addr 172.21.15.132:0/3031020591 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.148+0000 7f539b7fe640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 -- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 msgr2=0x56362bf09750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf09750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 -- 172.21.15.132:0/3031020591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 msgr2=0x56362bf09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 -- 172.21.15.132:0/3031020591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362be1dc60 con 0x56362befea10 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f539bfff640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f538c09b440 tx=0x7f538c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9723a70b6ecb9266 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b8070 con 0x56362befea10 2024-03-18T23:18:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f538c097db0 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c09f890 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f538c0b4040 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 10 client.25972 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56362bb77210 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c09f890 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f538c004120 con 0x56362befea10 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 10 client.25972 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.149+0000 7f53a51a0740 20 client.25972 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf1c740 0x56362bf1eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f53a51a0740 4 client.25972 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56362be70120 con 0x56362bf1c740 2024-03-18T23:18:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f539b7fe640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf1c740 0x56362bf1eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f537b7fe640 10 client.25972 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.150+0000 7f539b7fe640 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf1c740 0x56362bf1eb30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc0c2fba9817a9 server_cookie=bf2900993d2b8a25 in_seq=0 out_seq=0 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f537b7fe640 1 -- 172.21.15.132:0/3031020591 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56362be70120 con 0x56362bf1c740 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f537b7fe640 10 client.25972 handle_command_reply: tid=0 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 client.25972 shutdown 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 msgr2=0x56362bf0c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f538c09b440 tx=0x7f538c00ef90 comp rx=0 tx=0).stop 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 shutdown_connections 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf1c740 0x56362bf1eb30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362bf0c9f0 0x56362bf09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f537b7fe640 0 client.25972 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf09750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 --2- 172.21.15.132:0/3031020591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf0c3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 >> 172.21.15.132:0/3031020591 conn(0x56362bef9810 msgr2=0x56362befb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 shutdown_connections 2024-03-18T23:18:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 1 -- 172.21.15.132:0/3031020591 wait complete. 2024-03-18T23:18:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.151+0000 7f53a51a0740 20 client.25972 trim_cache size 0 max 16384 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 Processor -- start 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 -- start start 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf08800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362bf0c9f0 0x56362bf21b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1adcc0 con 0x56362befea10 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1ab590 con 0x56362beff330 2024-03-18T23:18:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a51a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1a9400 con 0x56362bf0c9f0 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54064/0 (socket says 172.21.15.132:54064) 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 -- 172.21.15.132:0/2094911250 learned_addr learned my addr 172.21.15.132:0/2094911250 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539b7fe640 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf08800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362bf0c9f0 0x56362bf21b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 -- 172.21.15.132:0/2094911250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362bf0c9f0 msgr2=0x56362bf21b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362bf0c9f0 0x56362bf21b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 -- 172.21.15.132:0/2094911250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 msgr2=0x56362bf08800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf08800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.166+0000 7f539bfff640 1 -- 172.21.15.132:0/2094911250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362bc43620 con 0x56362befea10 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f539bfff640 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f538c0cce30 tx=0x7f538c09b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8f2e4364104c09 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f5391ffb640 1 -- 172.21.15.132:0/2094911250 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0b8070 con 0x56362befea10 2024-03-18T23:18:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f5391ffb640 1 -- 172.21.15.132:0/2094911250 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c09be40 con 0x56362befea10 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f5391ffb640 1 -- 172.21.15.132:0/2094911250 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c004000 con 0x56362befea10 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 -- 172.21.15.132:0/2094911250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 msgr2=0x56362bef4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f538c0cce30 tx=0x7f538c09b0a0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 -- 172.21.15.132:0/2094911250 shutdown_connections 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362bf0c9f0 0x56362bf21b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362beff330 0x56362bf08800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 --2- 172.21.15.132:0/2094911250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362befea10 0x56362bef4940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 -- 172.21.15.132:0/2094911250 >> 172.21.15.132:0/2094911250 conn(0x56362bef9810 msgr2=0x56362bef9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 -- 172.21.15.132:0/2094911250 shutdown_connections 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.167+0000 7f53a51a0740 1 -- 172.21.15.132:0/2094911250 wait complete. 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 -- start start 2024-03-18T23:18:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 Processor -- start 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362bf0c9f0 0x56362bf1f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1a9a50 con 0x56362beff330 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1aaec0 con 0x56362bf0c9f0 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f539c1aa120 con 0x56362befea10 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a51a0740 10 client.0 _ll_register_callbacks cb 0x7f538c0b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f539bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54070/0 (socket says 172.21.15.132:54070) 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f539b7fe640 1 -- 172.21.15.132:0/1486423703 learned_addr learned my addr 172.21.15.132:0/1486423703 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.168+0000 7f53a0b4f640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362bf0c9f0 0x56362bf1f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 -- 172.21.15.132:0/1486423703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 msgr2=0x56362bf21c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf21c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 -- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362bf0c9f0 msgr2=0x56362bf1f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362bf0c9f0 0x56362bf1f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 -- 172.21.15.132:0/1486423703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362bf1fcd0 con 0x56362beff330 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539bfff640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f539b7fe640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad31a45df7785c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538402e020 con 0x56362beff330 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5384003d00 con 0x56362beff330 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f538c09be40 con 0x56362beff330 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5384039050 con 0x56362beff330 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f53a51a0740 10 client.16134 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56362bf825a0 con 0x56362beff330 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c09be40 con 0x56362beff330 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.169+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f538400fc10 con 0x56362beff330 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 10 client.16134 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 20 client.16134 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf82d50 0x56362bf85170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 4 client.16134 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56362be6a470 con 0x56362bf82d50 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f539bfff640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf82d50 0x56362bf85170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f537b7fe640 10 client.16134 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.170+0000 7f539bfff640 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf82d50 0x56362bf85170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c30fe113856d28 server_cookie=c6019ad30eb41596 in_seq=0 out_seq=0 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.177+0000 7f537b7fe640 1 -- 172.21.15.132:0/1486423703 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56362be6a470 con 0x56362bf82d50 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.177+0000 7f537b7fe640 10 client.16134 handle_command_reply: tid=0 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.177+0000 7f53a51a0740 1 client.16134 shutdown 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 msgr2=0x56362bf1f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384004730 comp rx=0 tx=0).stop 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 shutdown_connections 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56362bf82d50 0x56362bf85170 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56362bf0c9f0 0x56362bf1f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56362beff330 0x56362bf1f150 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f537b7fe640 0 client.16134 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 --2- 172.21.15.132:0/1486423703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56362befea10 0x56362bf21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 >> 172.21.15.132:0/1486423703 conn(0x56362bef9810 msgr2=0x56362bf04ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 shutdown_connections 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 1 -- 172.21.15.132:0/1486423703 wait complete. 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.178+0000 7f53a51a0740 20 client.16134 trim_cache size 0 max 16384 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:18:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 msgr2=0x7f53880a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 0x7f53880a4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f539c077bb0 tx=0x7f5384011860 comp rx=0 tx=0).stop 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 msgr2=0x7f539c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ea60 tx=0x7f538c00ef20 comp rx=0 tx=0).stop 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 shutdown_connections 2024-03-18T23:18:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f53880a2a10 0x7f53880a4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c1056c0 0x7f539c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c101330 0x7f539c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 --2- 172.21.15.132:0/1487033624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c100970 0x7f539c1981d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 >> 172.21.15.132:0/1487033624 conn(0x7f539c077570 msgr2=0x7f539c069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 shutdown_connections 2024-03-18T23:18:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:29.179+0000 7f53a25b2640 1 -- 172.21.15.132:0/1487033624 wait complete. 2024-03-18T23:18:29.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:28 smithi196 ceph-mon[67089]: pgmap v285: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:18:29.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:28 smithi144 ceph-mon[66781]: pgmap v285: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:18:29.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:28 smithi132 ceph-mon[32020]: pgmap v285: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:18:31.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:30 smithi196 ceph-mon[67089]: pgmap v286: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:18:31.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:30 smithi144 ceph-mon[66781]: pgmap v286: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:18:31.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:30 smithi132 ceph-mon[32020]: pgmap v286: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 9 op/s 2024-03-18T23:18:33.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:32 smithi196 ceph-mon[67089]: pgmap v287: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:33.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:32 smithi144 ceph-mon[66781]: pgmap v287: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:33.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:32 smithi132 ceph-mon[32020]: pgmap v287: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:35.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:34 smithi196 ceph-mon[67089]: pgmap v288: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:35.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:34 smithi144 ceph-mon[66781]: pgmap v288: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:35.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:34 smithi132 ceph-mon[32020]: pgmap v288: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:37.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:36 smithi196 ceph-mon[67089]: pgmap v289: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:37.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:36 smithi144 ceph-mon[66781]: pgmap v289: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:37.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:36 smithi132 ceph-mon[32020]: pgmap v289: 97 pgs: 97 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:39.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:38 smithi196 ceph-mon[67089]: pgmap v290: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:39.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:38 smithi144 ceph-mon[66781]: pgmap v290: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:39.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:38 smithi132 ceph-mon[32020]: pgmap v290: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:41.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:40 smithi196 ceph-mon[67089]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:41.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:40 smithi144 ceph-mon[66781]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:41.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:40 smithi132 ceph-mon[32020]: pgmap v291: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 -- 172.21.15.132:0/2535828550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3680fffd0 msgr2=0x7fe3681003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/2535828550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3680fffd0 0x7fe3681003b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe358009900 tx=0x7fe35802bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 -- 172.21.15.132:0/2535828550 shutdown_connections 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/2535828550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe368106be0 0x7fe36810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/2535828550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368100980 0x7fe3681064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/2535828550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3680fffd0 0x7fe3681003b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 -- 172.21.15.132:0/2535828550 >> 172.21.15.132:0/2535828550 conn(0x7fe368076c70 msgr2=0x7fe3680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 -- 172.21.15.132:0/2535828550 shutdown_connections 2024-03-18T23:18:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.273+0000 7fe36e9f4640 1 -- 172.21.15.132:0/2535828550 wait complete. 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 Processor -- start 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 -- start start 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fffd0 0x7fe368198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 0x7fe3681985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe368111c40 con 0x7fe36819c940 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe368111ac0 con 0x7fe368106be0 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36e9f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe368111dc0 con 0x7fe3680fffd0 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36cf91640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 0x7fe3681985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36cf91640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35638/0 (socket says 172.21.15.132:35638) 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36cf91640 1 -- 172.21.15.132:0/4290802690 learned_addr learned my addr 172.21.15.132:0/4290802690 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe367fff640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fffd0 0x7fe368198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.274+0000 7fe36cf91640 1 -- 172.21.15.132:0/4290802690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fffd0 msgr2=0x7fe368198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36cf91640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fffd0 0x7fe368198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36cf91640 1 -- 172.21.15.132:0/4290802690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 msgr2=0x7fe3681985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36cf91640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 0x7fe3681985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36cf91640 1 -- 172.21.15.132:0/4290802690 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36819d430 con 0x7fe36819c940 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 0x7fe3681985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36cf91640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe36000eb10 tx=0x7fe36000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe3657fa640 1 -- 172.21.15.132:0/4290802690 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004b20 con 0x7fe36819c940 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36819d720 con 0x7fe36819c940 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe3657fa640 1 -- 172.21.15.132:0/4290802690 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe360004cc0 con 0x7fe36819c940 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe3657fa640 1 -- 172.21.15.132:0/4290802690 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360010760 con 0x7fe36819c940 2024-03-18T23:18:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.275+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a4ec0 con 0x7fe36819c940 2024-03-18T23:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.277+0000 7fe3657fa640 1 -- 172.21.15.132:0/4290802690 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fe360005020 con 0x7fe36819c940 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 Processor -- start 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 -- start start 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16bea52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beb09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a16be08be0 con 0x55a16bea5810 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16bdb1960 con 0x55a16beae5e0 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3715e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a16be04c80 con 0x55a16bea4ef0 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3657fa640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 0x7fe3500a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe367fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe367fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35650/0 (socket says 172.21.15.132:35650) 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe367fff640 1 -- 172.21.15.132:0/4198592330 learned_addr learned my addr 172.21.15.132:0/4198592330 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beb09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16bea52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.278+0000 7fe36cf91640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 0x7fe3500a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3657fa640 1 -- 172.21.15.132:0/4290802690 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe3600cf370 con 0x7fe36819c940 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 -- 172.21.15.132:0/4198592330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 msgr2=0x55a16bea52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16bea52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 -- 172.21.15.132:0/4198592330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 msgr2=0x55a16beae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 -- 172.21.15.132:0/4198592330 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16bda8d20 con 0x55a16beae5e0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe3677fe640 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beb09d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe368073fa0 tx=0x7fe35c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686460cb0a8d8f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe36cf91640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 0x7fe3500a4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3600ccb70 tx=0x7fe36000efd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe34e7fc640 1 -- 172.21.15.132:0/4198592330 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011050 con 0x55a16beae5e0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe34e7fc640 1 -- 172.21.15.132:0/4198592330 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c0111f0 con 0x55a16beae5e0 2024-03-18T23:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.279+0000 7fe34e7fc640 1 -- 172.21.15.132:0/4198592330 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c005280 con 0x55a16beae5e0 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- 172.21.15.132:0/4198592330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 msgr2=0x55a16beb09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beb09d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe368073fa0 tx=0x7fe35c0046c0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- 172.21.15.132:0/4198592330 shutdown_connections 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beb09d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 --2- 172.21.15.132:0/4198592330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16bea52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- 172.21.15.132:0/4198592330 >> 172.21.15.132:0/4198592330 conn(0x55a16be9f810 msgr2=0x55a16be9fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- 172.21.15.132:0/4198592330 shutdown_connections 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- 172.21.15.132:0/4198592330 wait complete. 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 -- start start 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.280+0000 7fe3715e6740 1 Processor -- start 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16beadff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a16beb1210 con 0x55a16bea5810 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16bdc4db0 con 0x55a16beae5e0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a16beb1390 con 0x55a16bea4ef0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 10 client.0 _ll_register_callbacks cb 0x55a16bdc9e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35658/0 (socket says 172.21.15.132:35658) 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 -- 172.21.15.132:0/3313468044 learned_addr learned my addr 172.21.15.132:0/3313468044 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16beadff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 -- 172.21.15.132:0/3313468044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 msgr2=0x55a16beadff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16beadff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 -- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 msgr2=0x55a16beab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 -- 172.21.15.132:0/3313468044 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16bdc3c60 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe367fff640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe358009230 tx=0x7fe358009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1c9064c82734c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3477fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358004240 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe3580043e0 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.281+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35c0111f0 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe3580046c0 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 10 client.16146 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a16bb1d210 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe35c0111f0 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe3580035f0 con 0x55a16bea5810 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 10 client.16146 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 20 client.16146 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bebe5d0 0x55a16bec09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 4 client.16146 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16be16120 con 0x55a16bebe5d0 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bebe5d0 0x55a16bec09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3477fe640 10 client.16146 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.282+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bebe5d0 0x55a16bec09e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3305566cfac57a24 server_cookie=716ab6e4e471c9c7 in_seq=0 out_seq=0 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3477fe640 1 -- 172.21.15.132:0/3313468044 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a16be16120 con 0x55a16bebe5d0 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3477fe640 10 client.16146 handle_command_reply: tid=0 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 client.16146 shutdown 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 msgr2=0x55a16beab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe358009230 tx=0x7fe358009580 comp rx=0 tx=0).stop 2024-03-18T23:18:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 shutdown_connections 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bebe5d0 0x55a16bec09e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16beae5e0 0x55a16beab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea5810 0x55a16beab380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3477fe640 0 client.16146 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3313468044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea4ef0 0x55a16beadff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 >> 172.21.15.132:0/3313468044 conn(0x55a16be9f810 msgr2=0x55a16be9eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 shutdown_connections 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 1 -- 172.21.15.132:0/3313468044 wait complete. 2024-03-18T23:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.283+0000 7fe3715e6740 20 client.16146 trim_cache size 0 max 16384 2024-03-18T23:18:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 Processor -- start 2024-03-18T23:18:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 -- start start 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea4ef0 0x55a16bec09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 0x55a16beb9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681afcf0 con 0x55a16beae5e0 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681ad5c0 con 0x55a16bea4ef0 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.298+0000 7fe3715e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681ab430 con 0x55a16bea5810 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35664/0 (socket says 172.21.15.132:35664) 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe3677fe640 1 -- 172.21.15.132:0/3512180956 learned_addr learned my addr 172.21.15.132:0/3512180956 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 0x55a16beb9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea4ef0 0x55a16bec09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 -- 172.21.15.132:0/3512180956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea4ef0 msgr2=0x55a16bec09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea4ef0 0x55a16bec09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 -- 172.21.15.132:0/3512180956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 msgr2=0x55a16bec62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 -- 172.21.15.132:0/3512180956 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16bbe9620 con 0x55a16bea5810 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe367fff640 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 0x55a16beb9fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe35802c170 tx=0x7fe35802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ef7a0eaab275c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe34dffb640 1 -- 172.21.15.132:0/3512180956 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35802cc20 con 0x55a16bea5810 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe34dffb640 1 -- 172.21.15.132:0/3512180956 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe358004360 con 0x55a16bea5810 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.299+0000 7fe34dffb640 1 -- 172.21.15.132:0/3512180956 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358048830 con 0x55a16bea5810 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- 172.21.15.132:0/3512180956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 msgr2=0x55a16beb9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 0x55a16beb9fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe35802c170 tx=0x7fe35802bc30 comp rx=0 tx=0).stop 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- 172.21.15.132:0/3512180956 shutdown_connections 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16beae5e0 0x55a16bec62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16bea5810 0x55a16beb9fb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3512180956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea4ef0 0x55a16bec09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- 172.21.15.132:0/3512180956 >> 172.21.15.132:0/3512180956 conn(0x55a16be9f810 msgr2=0x55a16be9eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- 172.21.15.132:0/3512180956 shutdown_connections 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- 172.21.15.132:0/3512180956 wait complete. 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 -- start start 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.300+0000 7fe3715e6740 1 Processor -- start 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 0x55a16bea8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16beae5e0 0x55a16bea8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681aba80 con 0x55a16bea4ef0 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681acef0 con 0x55a16bea5810 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681ac150 con 0x55a16beae5e0 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 10 client.0 _ll_register_callbacks cb 0x7fe358009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:35674/0 (socket says 172.21.15.132:35674) 2024-03-18T23:18:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 -- 172.21.15.132:0/3337149389 learned_addr learned my addr 172.21.15.132:0/3337149389 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe367fff640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 0x55a16bea8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16beae5e0 0x55a16bea8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 -- 172.21.15.132:0/3337149389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16beae5e0 msgr2=0x55a16bea8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16beae5e0 0x55a16bea8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 -- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 msgr2=0x55a16bea8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 0x55a16bea8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 -- 172.21.15.132:0/3337149389 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16bf23a20 con 0x55a16bea4ef0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe367fff640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 0x55a16bea8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3677fe640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe35c013fd0 tx=0x7fe35c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a75daf1f54f8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3477fe640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c030730 con 0x55a16bea4ef0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe358004360 con 0x55a16bea4ef0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fe35c004030 con 0x55a16bea4ef0 2024-03-18T23:18:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe35c040050 con 0x55a16bea4ef0 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 10 client.16152 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.301+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a16bf24d30 con 0x55a16bea4ef0 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe358004360 con 0x55a16bea4ef0 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fe35c02f040 con 0x55a16bea4ef0 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 10 client.16152 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 20 client.16152 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bf25290 0x55a16bf276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 4 client.16152 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a16be10470 con 0x55a16bf25290 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe367fff640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bf25290 0x55a16bf276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe3477fe640 10 client.16152 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.302+0000 7fe367fff640 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bf25290 0x55a16bf276b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0bd51627761b074 server_cookie=bb87dd1d3b6ed094 in_seq=0 out_seq=0 2024-03-18T23:18:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.309+0000 7fe3477fe640 1 -- 172.21.15.132:0/3337149389 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a16be10470 con 0x55a16bf25290 2024-03-18T23:18:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.309+0000 7fe3477fe640 10 client.16152 handle_command_reply: tid=0 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.309+0000 7fe3715e6740 1 client.16152 shutdown 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.309+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 msgr2=0x55a16bea8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.309+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe35c013fd0 tx=0x7fe35c005f90 comp rx=0 tx=0).stop 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 shutdown_connections 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55a16bf25290 0x55a16bf276b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a16beae5e0 0x55a16bea8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16bea5810 0x55a16bea8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3477fe640 0 client.16152 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 --2- 172.21.15.132:0/3337149389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a16bea4ef0 0x55a16bea8020 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 >> 172.21.15.132:0/3337149389 conn(0x55a16be9f810 msgr2=0x55a16bea0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 shutdown_connections 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 1 -- 172.21.15.132:0/3337149389 wait complete. 2024-03-18T23:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe3715e6740 20 client.16152 trim_cache size 0 max 16384 2024-03-18T23:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 msgr2=0x7fe3500a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 0x7fe3500a4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3600ccb70 tx=0x7fe36000efd0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 msgr2=0x7fe36819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe36000eb10 tx=0x7fe36000c2d0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 shutdown_connections 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fe3500a2a10 0x7fe3500a4ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe36819c940 0x7fe36819cdf0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe368106be0 0x7fe3681985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 --2- 172.21.15.132:0/4290802690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3680fffd0 0x7fe368198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 >> 172.21.15.132:0/4290802690 conn(0x7fe368076c70 msgr2=0x7fe368104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 shutdown_connections 2024-03-18T23:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:42.310+0000 7fe36e9f4640 1 -- 172.21.15.132:0/4290802690 wait complete. 2024-03-18T23:18:43.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:42 smithi196 ceph-mon[67089]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:18:43.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:42 smithi144 ceph-mon[66781]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:18:43.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:42 smithi132 ceph-mon[32020]: pgmap v292: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-03-18T23:18:45.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:44 smithi196 ceph-mon[67089]: pgmap v293: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:45.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:44 smithi144 ceph-mon[66781]: pgmap v293: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:45.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:44 smithi132 ceph-mon[32020]: pgmap v293: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:47.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:46 smithi196 ceph-mon[67089]: pgmap v294: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:47.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:46 smithi144 ceph-mon[66781]: pgmap v294: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:47.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:46 smithi132 ceph-mon[32020]: pgmap v294: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 9 op/s 2024-03-18T23:18:49.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:48 smithi196 ceph-mon[67089]: pgmap v295: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:49.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:48 smithi144 ceph-mon[66781]: pgmap v295: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:49.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:48 smithi132 ceph-mon[32020]: pgmap v295: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-03-18T23:18:51.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:50 smithi196 ceph-mon[67089]: pgmap v296: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:51.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:50 smithi144 ceph-mon[66781]: pgmap v296: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:51.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:50 smithi132 ceph-mon[32020]: pgmap v296: 97 pgs: 97 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:53.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:52 smithi196 ceph-mon[67089]: pgmap v297: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:53.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:52 smithi144 ceph-mon[66781]: pgmap v297: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:53.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:52 smithi132 ceph-mon[32020]: pgmap v297: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:18:55.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:54 smithi144 ceph-mon[66781]: pgmap v298: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:55.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:54 smithi132 ceph-mon[32020]: pgmap v298: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 -- 172.21.15.132:0/375612658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81021b0 msgr2=0x7feeb810e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 --2- 172.21.15.132:0/375612658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81021b0 0x7feeb810e6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009be0 tx=0x7feeb002bef0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 -- 172.21.15.132:0/375612658 shutdown_connections 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 --2- 172.21.15.132:0/375612658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81021b0 0x7feeb810e6c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 --2- 172.21.15.132:0/375612658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb81017f0 0x7feeb8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 --2- 172.21.15.132:0/375612658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb8107800 0x7feeb8107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 -- 172.21.15.132:0/375612658 >> 172.21.15.132:0/375612658 conn(0x7feeb80fd300 msgr2=0x7feeb80ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 -- 172.21.15.132:0/375612658 shutdown_connections 2024-03-18T23:18:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.407+0000 7feebcee8640 1 -- 172.21.15.132:0/375612658 wait complete. 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 Processor -- start 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 -- start start 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 0x7feeb8073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 0x7feeb80745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8111bd0 con 0x7feeb81021b0 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8111a50 con 0x7feeb81017f0 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feebcee8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8111d50 con 0x7feeb8107800 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feeb6d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 0x7feeb80745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55056/0 (socket says 172.21.15.132:55056) 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feeb659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 0x7feeb8073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.408+0000 7feeb5d9b640 1 -- 172.21.15.132:0/1386745384 learned_addr learned my addr 172.21.15.132:0/1386745384 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 -- 172.21.15.132:0/1386745384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 msgr2=0x7feeb8073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 0x7feeb8073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 -- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 msgr2=0x7feeb8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 -- 172.21.15.132:0/1386745384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb81a6470 con 0x7feeb8107800 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb659c640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 0x7feeb8073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 0x7feeb80745a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feeb0033420 tx=0x7feeb00336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeaf7fe640 1 -- 172.21.15.132:0/1386745384 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0043020 con 0x7feeb8107800 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb81a6700 con 0x7feeb8107800 2024-03-18T23:18:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeaf7fe640 1 -- 172.21.15.132:0/1386745384 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0003e50 con 0x7feeb8107800 2024-03-18T23:18:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feeaf7fe640 1 -- 172.21.15.132:0/1386745384 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0034430 con 0x7feeb8107800 2024-03-18T23:18:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.409+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a6c50 con 0x7feeb8107800 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.411+0000 7feeaf7fe640 1 -- 172.21.15.132:0/1386745384 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7feeb0046020 con 0x7feeb8107800 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 Processor -- start 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 -- start start 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feeaf7fe640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 0x7feea40a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x5601831180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 0x56018311a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x560183072be0 con 0x56018310eef0 2024-03-18T23:18:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56018301b960 con 0x5601831185e0 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feebfba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56018306ec80 con 0x56018310f810 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 0x7feea40a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feeb659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.412+0000 7feeaf7fe640 1 -- 172.21.15.132:0/1386745384 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7feeb00f11d0 con 0x7feeb8107800 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55060/0 (socket says 172.21.15.132:55060) 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 -- 172.21.15.132:0/3400337432 learned_addr learned my addr 172.21.15.132:0/3400337432 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x5601831180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 0x56018311a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 -- 172.21.15.132:0/3400337432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 msgr2=0x5601831180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x5601831180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 -- 172.21.15.132:0/3400337432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 msgr2=0x56018311a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 0x56018311a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 -- 172.21.15.132:0/3400337432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560183012d20 con 0x56018310eef0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 0x56018311a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb659c640 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feea0011470 tx=0x7feea0011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bc9b3dcdf8b084 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 0x7feea40a4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feea80098e0 tx=0x7feea8009d00 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeacff9640 1 -- 172.21.15.132:0/3400337432 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0005020 con 0x56018310eef0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeacff9640 1 -- 172.21.15.132:0/3400337432 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea00047a0 con 0x56018310eef0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.413+0000 7feeacff9640 1 -- 172.21.15.132:0/3400337432 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea000f820 con 0x56018310eef0 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- 172.21.15.132:0/3400337432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 msgr2=0x56018310f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feea0011470 tx=0x7feea0011870 comp rx=0 tx=0).stop 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- 172.21.15.132:0/3400337432 shutdown_connections 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601831185e0 0x56018311a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x5601831180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 --2- 172.21.15.132:0/3400337432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310eef0 0x56018310f2d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- 172.21.15.132:0/3400337432 >> 172.21.15.132:0/3400337432 conn(0x560183109810 msgr2=0x560183109bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- 172.21.15.132:0/3400337432 shutdown_connections 2024-03-18T23:18:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- 172.21.15.132:0/3400337432 wait complete. 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 -- start start 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.414+0000 7feebfba9740 1 Processor -- start 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183115790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x56018311b210 con 0x5601831185e0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56018302edb0 con 0x56018310f810 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56018311b390 con 0x56018310eef0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feebfba9740 10 client.0 _ll_register_callbacks cb 0x560183033e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183115790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57230/0 (socket says 172.21.15.132:57230) 2024-03-18T23:18:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:34258/0 (socket says 172.21.15.132:34258) 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 -- 172.21.15.132:0/3460403444 learned_addr learned my addr 172.21.15.132:0/3460403444 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 -- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 msgr2=0x560183115250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 -- 172.21.15.132:0/3460403444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 msgr2=0x560183115790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183115790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 -- 172.21.15.132:0/3460403444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018302dc60 con 0x56018310eef0 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7feeb659c640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feea0005a40 tx=0x7feea000fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c29c9acc64ebba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.415+0000 7fee9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0028070 con 0x56018310eef0 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7feea0004020 con 0x56018310eef0 2024-03-18T23:18:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea00047a0 con 0x56018310eef0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feea0009c40 con 0x56018310eef0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 10 client.26008 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560182d87210 con 0x56018310eef0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea00047a0 con 0x56018310eef0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7feea00035f0 con 0x56018310eef0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 10 client.26008 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 10 client.26008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 20 client.26008 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5601831282f0 0x56018312a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 4 client.26008 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560183080120 con 0x5601831282f0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.416+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5601831282f0 0x56018312a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5601831282f0 0x56018312a6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6dc4d12d737f7e server_cookie=af6272b5132a8869 in_seq=0 out_seq=0 2024-03-18T23:18:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7fee9dffb640 10 client.26008 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7fee9dffb640 1 -- 172.21.15.132:0/3460403444 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560183080120 con 0x5601831282f0 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7fee9dffb640 10 client.26008 handle_command_reply: tid=0 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 client.26008 shutdown 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 msgr2=0x560183117ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feea0005a40 tx=0x7feea000fca0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 shutdown_connections 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x5601831282f0 0x56018312a6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310f810 0x560183115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7fee9dffb640 0 client.26008 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.417+0000 7feebfba9740 1 --2- 172.21.15.132:0/3460403444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310eef0 0x560183117ec0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.418+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 >> 172.21.15.132:0/3460403444 conn(0x560183109810 msgr2=0x560183108ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.418+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 shutdown_connections 2024-03-18T23:18:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.418+0000 7feebfba9740 1 -- 172.21.15.132:0/3460403444 wait complete. 2024-03-18T23:18:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.418+0000 7feebfba9740 20 client.26008 trim_cache size 0 max 16384 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 Processor -- start 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 -- start start 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x560183123fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601831185e0 0x560183117980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81b1f00 con 0x56018310f810 2024-03-18T23:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81af7d0 con 0x56018310eef0 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.432+0000 7feebfba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ad640 con 0x5601831185e0 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x560183123fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:55068/0 (socket says 172.21.15.132:55068) 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 -- 172.21.15.132:0/4283870377 learned_addr learned my addr 172.21.15.132:0/4283870377 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601831185e0 0x560183117980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 -- 172.21.15.132:0/4283870377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601831185e0 msgr2=0x560183117980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601831185e0 0x560183117980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 -- 172.21.15.132:0/4283870377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 msgr2=0x560183123fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x560183123fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 -- 172.21.15.132:0/4283870377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018310e720 con 0x56018310f810 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb659c640 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x560183123fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7feea8009710 tx=0x7feea8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c732743c66194e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7fee9ffff640 1 -- 172.21.15.132:0/4283870377 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8032070 con 0x56018310f810 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7fee9ffff640 1 -- 172.21.15.132:0/4283870377 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea80044d0 con 0x56018310f810 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.433+0000 7fee9ffff640 1 -- 172.21.15.132:0/4283870377 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8019040 con 0x56018310f810 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- 172.21.15.132:0/4283870377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 msgr2=0x56018312fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7feea8009710 tx=0x7feea8020000 comp rx=0 tx=0).stop 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- 172.21.15.132:0/4283870377 shutdown_connections 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601831185e0 0x560183117980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018310f810 0x56018312fe50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- 172.21.15.132:0/4283870377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x560183123fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- 172.21.15.132:0/4283870377 >> 172.21.15.132:0/4283870377 conn(0x560183109810 msgr2=0x560183115210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- 172.21.15.132:0/4283870377 shutdown_connections 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- 172.21.15.132:0/4283870377 wait complete. 2024-03-18T23:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- start start 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 Processor -- start 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x5601831205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183111a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81adc90 con 0x5601831185e0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81af100 con 0x56018310eef0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ae360 con 0x56018310f810 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feebfba9740 10 client.0 _ll_register_callbacks cb 0x7feea8010550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feeb6d9d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183111a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feeb5d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:57252/0 (socket says 172.21.15.132:57252) 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.434+0000 7feeb5d9b640 1 -- 172.21.15.132:0/403785977 learned_addr learned my addr 172.21.15.132:0/403785977 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb659c640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x5601831205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 -- 172.21.15.132:0/403785977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 msgr2=0x560183111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 -- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 msgr2=0x5601831205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x5601831205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 -- 172.21.15.132:0/403785977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560182e53620 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb5d9b640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feeb6d9d640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183111a60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feeb00eea30 tx=0x7feeb00bd360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2344fefe006e56e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7fee9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00c5e50 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7feeb00a8070 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea80044d0 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feeb00be040 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feebfba9740 10 client.16170 fetch_fsmap learned FSMap version 24 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56018318dd30 con 0x5601831185e0 2024-03-18T23:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.435+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea80044d0 con 0x5601831185e0 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7feeb0004b40 con 0x5601831185e0 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 10 client.16170 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 20 client.16170 populate_metadata read hostname 'smithi132' 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56018318e400 0x560183190820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 4 client.16170 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56018307a470 con 0x56018318e400 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feeb659c640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56018318e400 0x560183190820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7feeb659c640 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56018318e400 0x560183190820 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8fa4d42ade6bb5d server_cookie=f290a503224406d0 in_seq=0 out_seq=0 2024-03-18T23:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.436+0000 7fee9dffb640 10 client.16170 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.443+0000 7fee9dffb640 1 -- 172.21.15.132:0/403785977 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56018307a470 con 0x56018318e400 2024-03-18T23:18:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.443+0000 7fee9dffb640 10 client.16170 handle_command_reply: tid=0 2024-03-18T23:18:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.443+0000 7feebfba9740 1 client.16170 shutdown 2024-03-18T23:18:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 msgr2=0x560183111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183111a60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feeb00eea30 tx=0x7feeb00bd360 comp rx=0 tx=0).stop 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 shutdown_connections 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x56018318e400 0x560183190820 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601831185e0 0x560183111a60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56018310f810 0x560183111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7fee9dffb640 0 client.16170 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 --2- 172.21.15.132:0/403785977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56018310eef0 0x5601831205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 >> 172.21.15.132:0/403785977 conn(0x560183109810 msgr2=0x56018310ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 shutdown_connections 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 1 -- 172.21.15.132:0/403785977 wait complete. 2024-03-18T23:18:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebfba9740 20 client.16170 trim_cache size 0 max 16384 2024-03-18T23:18:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:18:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:18:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:18:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 msgr2=0x7feea40a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 0x7feea40a4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feea80098e0 tx=0x7feea8009d00 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 msgr2=0x7feeb80745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 0x7feeb80745a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feeb0033420 tx=0x7feeb00336b0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 shutdown_connections 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7feea40a2a40 0x7feea40a4f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107800 0x7feeb80745a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feeb81021b0 0x7feeb8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 --2- 172.21.15.132:0/1386745384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feeb81017f0 0x7feeb8073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:18:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 >> 172.21.15.132:0/1386745384 conn(0x7feeb80fd300 msgr2=0x7feeb80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:18:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 shutdown_connections 2024-03-18T23:18:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:18:55.444+0000 7feebcee8640 1 -- 172.21.15.132:0/1386745384 wait complete. 2024-03-18T23:18:55.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:54 smithi196 ceph-mon[67089]: pgmap v298: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:57.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:56 smithi144 ceph-mon[66781]: pgmap v299: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:57.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:56 smithi132 ceph-mon[32020]: pgmap v299: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:57.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:56 smithi196 ceph-mon[67089]: pgmap v299: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:18:59.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:18:58 smithi144 ceph-mon[66781]: pgmap v300: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:18:59.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:18:58 smithi132 ceph-mon[32020]: pgmap v300: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:18:59.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:18:58 smithi196 ceph-mon[67089]: pgmap v300: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:19:01.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:00 smithi144 ceph-mon[66781]: pgmap v301: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:01.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:00 smithi132 ceph-mon[32020]: pgmap v301: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:01.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:00 smithi196 ceph-mon[67089]: pgmap v301: 97 pgs: 97 active+clean; 5.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:03.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:02 smithi144 ceph-mon[66781]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:19:03.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:02 smithi132 ceph-mon[32020]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:19:03.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:02 smithi196 ceph-mon[67089]: pgmap v302: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-03-18T23:19:05.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:04 smithi144 ceph-mon[66781]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:19:05.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:04 smithi132 ceph-mon[32020]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:19:05.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:04 smithi196 ceph-mon[67089]: pgmap v303: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-03-18T23:19:06.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:05 smithi144 ceph-mon[66781]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:19:06.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:05 smithi132 ceph-mon[32020]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:19:06.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:05 smithi196 ceph-mon[67089]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T23:19:07.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:07 smithi144 ceph-mon[66781]: pgmap v305: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:19:07.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:07 smithi132 ceph-mon[32020]: pgmap v305: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:19:07.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:07 smithi196 ceph-mon[67089]: pgmap v305: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-18T23:19:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 -- 172.21.15.132:0/1704424766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 msgr2=0x7f9ab8105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1704424766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab8105aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009900 tx=0x7f9ab402bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 -- 172.21.15.132:0/1704424766 shutdown_connections 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1704424766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1704424766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.541+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1704424766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab8105aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- 172.21.15.132:0/1704424766 >> 172.21.15.132:0/1704424766 conn(0x7f9ab8077570 msgr2=0x7f9ab8077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- 172.21.15.132:0/1704424766 shutdown_connections 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- 172.21.15.132:0/1704424766 wait complete. 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 Processor -- start 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- start start 2024-03-18T23:19:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8111d00 con 0x7f9ab81056c0 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8111b80 con 0x7f9ab8100970 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.542+0000 7f9abfc63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab8111e80 con 0x7f9ab8101330 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51006/0 (socket says 172.21.15.132:51006) 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 -- 172.21.15.132:0/1915876691 learned_addr learned my addr 172.21.15.132:0/1915876691 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 -- 172.21.15.132:0/1915876691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 msgr2=0x7f9ab8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 -- 172.21.15.132:0/1915876691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 msgr2=0x7f9ab8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 -- 172.21.15.132:0/1915876691 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab819d0a0 con 0x7f9ab81056c0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abe200640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000ec70 tx=0x7f9ab000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9aaeffd640 1 -- 172.21.15.132:0/1915876691 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0003b70 con 0x7f9ab81056c0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9aaeffd640 1 -- 172.21.15.132:0/1915876691 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab000ee80 con 0x7f9ab81056c0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab819d390 con 0x7f9ab81056c0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9aaeffd640 1 -- 172.21.15.132:0/1915876691 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00108a0 con 0x7f9ab81056c0 2024-03-18T23:19:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.543+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab81a4c20 con 0x7f9ab81056c0 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.545+0000 7f9aaeffd640 1 -- 172.21.15.132:0/1915876691 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f9ab0004020 con 0x7f9ab81056c0 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.546+0000 7f9ac2849740 1 Processor -- start 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.546+0000 7f9aaeffd640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 0x7f9aa40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.546+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 0x7f9aa40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.546+0000 7f9aaeffd640 1 -- 172.21.15.132:0/1915876691 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f9ab00cf240 con 0x7f9ab81056c0 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 -- start start 2024-03-18T23:19:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0dae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 0x55dba0db99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55dba0d11be0 con 0x55dba0db75e0 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba0cba960 con 0x55dba0dadef0 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9ac2849740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dba0d0dc80 con 0x55dba0dae810 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abe200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 0x55dba0db99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abe200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 0x55dba0db99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51012/0 (socket says 172.21.15.132:51012) 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:47060/0 (socket says 172.21.15.132:47060) 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 0x7f9aa40a4ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402efd0 tx=0x7f9ab4000bc0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abe200640 1 -- 172.21.15.132:0/1506307013 learned_addr learned my addr 172.21.15.132:0/1506307013 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0dae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 -- 172.21.15.132:0/1506307013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 msgr2=0x55dba0dae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0dae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 -- 172.21.15.132:0/1506307013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 msgr2=0x55dba0db99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 0x55dba0db99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 -- 172.21.15.132:0/1506307013 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba0cb1d20 con 0x55dba0dae810 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.547+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009fa0 tx=0x7f9aa80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73085a2f0691dcd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9aa3fff640 1 -- 172.21.15.132:0/1506307013 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8004020 con 0x55dba0dae810 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9aa3fff640 1 -- 172.21.15.132:0/1506307013 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa80041c0 con 0x55dba0dae810 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9aa3fff640 1 -- 172.21.15.132:0/1506307013 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800f910 con 0x55dba0dae810 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 -- 172.21.15.132:0/1506307013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 msgr2=0x55dba0db70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009fa0 tx=0x7f9aa80118d0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 -- 172.21.15.132:0/1506307013 shutdown_connections 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0db75e0 0x55dba0db99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dae810 0x55dba0db70a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1506307013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0dae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.548+0000 7f9ac2849740 1 -- 172.21.15.132:0/1506307013 >> 172.21.15.132:0/1506307013 conn(0x55dba0da8810 msgr2=0x55dba0da8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- 172.21.15.132:0/1506307013 shutdown_connections 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- 172.21.15.132:0/1506307013 wait complete. 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- start start 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 Processor -- start 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55dba0dba210 con 0x55dba0dae810 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba0ccddb0 con 0x55dba0dadef0 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dba0dba390 con 0x55dba0db75e0 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.549+0000 7f9ac2849740 10 client.0 _ll_register_callbacks cb 0x55dba0cd2e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:59444/0 (socket says 172.21.15.132:59444) 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 -- 172.21.15.132:0/1322178064 learned_addr learned my addr 172.21.15.132:0/1322178064 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abe200640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 -- 172.21.15.132:0/1322178064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 msgr2=0x55dba0db48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 -- 172.21.15.132:0/1322178064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 msgr2=0x55dba0db4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 -- 172.21.15.132:0/1322178064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba0cccc60 con 0x55dba0dadef0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402c850 tx=0x7f9ab402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5452a711ab639ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9aa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004f80 con 0x55dba0dadef0 2024-03-18T23:19:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa80041c0 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f9ab4005120 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9ab4005320 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9ac2849740 10 client.26027 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.550+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dba0a26210 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa80041c0 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9ab4034440 con 0x55dba0dadef0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 10 client.26027 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 20 client.26027 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0dc7470 0x55dba0dc9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 4 client.26027 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba0d1f120 con 0x55dba0dc7470 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0dc7470 0x55dba0dc9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0dc7470 0x55dba0dc9860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a711480e269fcc server_cookie=ad487099a891d7ce in_seq=0 out_seq=0 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.551+0000 7f9aa17fa640 10 client.26027 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9aa17fa640 1 -- 172.21.15.132:0/1322178064 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dba0d1f120 con 0x55dba0dc7470 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9aa17fa640 10 client.26027 handle_command_reply: tid=0 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 client.26027 shutdown 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 msgr2=0x55dba0db6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402c850 tx=0x7f9ab402bc30 comp rx=0 tx=0).stop 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 shutdown_connections 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0dc7470 0x55dba0dc9860 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 --2- 172.21.15.132:0/1322178064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db6f80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9aa17fa640 0 client.26027 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 >> 172.21.15.132:0/1322178064 conn(0x55dba0da8810 msgr2=0x55dba0da7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 shutdown_connections 2024-03-18T23:19:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.552+0000 7f9ac2849740 1 -- 172.21.15.132:0/1322178064 wait complete. 2024-03-18T23:19:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.553+0000 7f9ac2849740 20 client.26027 trim_cache size 0 max 16384 2024-03-18T23:19:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 Processor -- start 2024-03-18T23:19:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 -- start start 2024-03-18T23:19:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0dc2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0db75e0 0x55dba0da7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81afd90 con 0x55dba0dae810 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81ad660 con 0x55dba0db75e0 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9ac2849740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81ab4d0 con 0x55dba0dadef0 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0dc2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:47080/0 (socket says 172.21.15.132:47080) 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abd9ff640 1 -- 172.21.15.132:0/2727086599 learned_addr learned my addr 172.21.15.132:0/2727086599 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0dc2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51016/0 (socket says 172.21.15.132:51016) 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.567+0000 7f9abe200640 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0db75e0 0x55dba0da7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 -- 172.21.15.132:0/2727086599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0db75e0 msgr2=0x55dba0da7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0db75e0 0x55dba0da7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 -- 172.21.15.132:0/2727086599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 msgr2=0x55dba0dc2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0dc2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 -- 172.21.15.132:0/2727086599 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba0db4c20 con 0x55dba0dadef0 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402c240 tx=0x7f9ab4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4772e8fa76d0dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9aa37fe640 1 -- 172.21.15.132:0/2727086599 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4034e70 con 0x55dba0dadef0 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9aa37fe640 1 -- 172.21.15.132:0/2727086599 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab402cc40 con 0x55dba0dadef0 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.568+0000 7f9aa37fe640 1 -- 172.21.15.132:0/2727086599 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40054c0 con 0x55dba0dadef0 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- 172.21.15.132:0/2727086599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 msgr2=0x55dba0dbf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402c240 tx=0x7f9ab4009680 comp rx=0 tx=0).stop 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- 172.21.15.132:0/2727086599 shutdown_connections 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0db75e0 0x55dba0da7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0dc2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- 172.21.15.132:0/2727086599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0dadef0 0x55dba0dbf9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- 172.21.15.132:0/2727086599 >> 172.21.15.132:0/2727086599 conn(0x55dba0da8810 msgr2=0x55dba0da7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- 172.21.15.132:0/2727086599 shutdown_connections 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- 172.21.15.132:0/2727086599 wait complete. 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- start start 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 Processor -- start 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81abb20 con 0x55dba0dae810 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81acf90 con 0x55dba0dadef0 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab81ac1f0 con 0x55dba0db75e0 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9ac2849740 10 client.0 _ll_register_callbacks cb 0x7f9ab00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.569+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:59450/0 (socket says 172.21.15.132:59450) 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 -- 172.21.15.132:0/4250930653 learned_addr learned my addr 172.21.15.132:0/4250930653 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd1fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:51022/0 (socket says 172.21.15.132:51022) 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abe200640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 -- 172.21.15.132:0/4250930653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 msgr2=0x55dba0db1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 -- 172.21.15.132:0/4250930653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 msgr2=0x55dba0db0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 -- 172.21.15.132:0/4250930653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba0dae640 con 0x55dba0dadef0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9abd9ff640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402f020 tx=0x7f9ab4034680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc1395eda3788d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9aa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4069020 con 0x55dba0dadef0 2024-03-18T23:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f9ab40054c0 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab402cc40 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9ab4074050 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9ac2849740 10 client.26033 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.570+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dba0af2620 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab402cc40 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9ab4065480 con 0x55dba0dadef0 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 10 client.26033 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 20 client.26033 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0e31f70 0x55dba0e34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 4 client.26033 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dba0d19470 con 0x55dba0e31f70 2024-03-18T23:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0e31f70 0x55dba0e34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9abd1fe640 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0e31f70 0x55dba0e34390 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b76ff3735758 server_cookie=a196c383dba38173 in_seq=0 out_seq=0 2024-03-18T23:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.571+0000 7f9aa17fa640 10 client.26033 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9aa17fa640 1 -- 172.21.15.132:0/4250930653 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dba0d19470 con 0x55dba0e31f70 2024-03-18T23:19:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9aa17fa640 10 client.26033 handle_command_reply: tid=0 2024-03-18T23:19:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 client.26033 shutdown 2024-03-18T23:19:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 msgr2=0x55dba0db05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402f020 tx=0x7f9ab4034680 comp rx=0 tx=0).stop 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 shutdown_connections 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55dba0e31f70 0x55dba0e34390 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dba0db75e0 0x55dba0db1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dba0dae810 0x55dba0db0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9aa17fa640 0 client.26033 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 --2- 172.21.15.132:0/4250930653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0dadef0 0x55dba0db05c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 >> 172.21.15.132:0/4250930653 conn(0x55dba0da8810 msgr2=0x55dba0da7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 shutdown_connections 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 1 -- 172.21.15.132:0/4250930653 wait complete. 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.579+0000 7f9ac2849740 20 client.26033 trim_cache size 0 max 16384 2024-03-18T23:19:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:19:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:19:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:19:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:19:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 msgr2=0x7f9aa40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 0x7f9aa40a4ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402efd0 tx=0x7f9ab4000bc0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 msgr2=0x7f9ab819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000ec70 tx=0x7f9ab000c6a0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 shutdown_connections 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f9aa40a2a10 0x7f9aa40a4ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ab81056c0 0x7f9ab819c970 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ab8101330 0x7f9ab8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 --2- 172.21.15.132:0/1915876691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8100970 0x7f9ab8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 >> 172.21.15.132:0/1915876691 conn(0x7f9ab8077570 msgr2=0x7f9ab80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 shutdown_connections 2024-03-18T23:19:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:08.580+0000 7f9abfc63640 1 -- 172.21.15.132:0/1915876691 wait complete. 2024-03-18T23:19:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:09 smithi144 ceph-mon[66781]: pgmap v306: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-03-18T23:19:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:09 smithi144 ceph-mon[66781]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:19:09.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:09 smithi132 ceph-mon[32020]: pgmap v306: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-03-18T23:19:09.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:09 smithi132 ceph-mon[32020]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:19:09.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:09 smithi196 ceph-mon[67089]: pgmap v306: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-03-18T23:19:09.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:09 smithi196 ceph-mon[67089]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T23:19:11.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:11 smithi144 ceph-mon[66781]: pgmap v308: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:19:11.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:11 smithi132 ceph-mon[32020]: pgmap v308: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:19:11.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:11 smithi196 ceph-mon[67089]: pgmap v308: 97 pgs: 97 active+clean; 5.4 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2024-03-18T23:19:13.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:13 smithi144 ceph-mon[66781]: pgmap v309: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:19:13.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:13 smithi132 ceph-mon[32020]: pgmap v309: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:19:13.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:13 smithi196 ceph-mon[67089]: pgmap v309: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-03-18T23:19:15.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:15 smithi144 ceph-mon[66781]: pgmap v310: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:19:15.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:15 smithi132 ceph-mon[32020]: pgmap v310: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:19:15.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:15 smithi196 ceph-mon[67089]: pgmap v310: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-18T23:19:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:17 smithi144 ceph-mon[66781]: pgmap v311: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:19:17.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:17 smithi132 ceph-mon[32020]: pgmap v311: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:19:17.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:17 smithi196 ceph-mon[67089]: pgmap v311: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-03-18T23:19:19.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:19 smithi144 ceph-mon[66781]: pgmap v312: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-18T23:19:19.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:19 smithi132 ceph-mon[32020]: pgmap v312: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-18T23:19:19.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:19 smithi196 ceph-mon[67089]: pgmap v312: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:499/600: open fd = 3 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:500/600: open fd = 3 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.551 INFO:tasks.workunit.client.0.smithi132.stdout:501/600: open fd = 3 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:502/600: open fd = 3 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:503/600: open fd = 3 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:504/600: open fd = 3 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.552 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:505/600: open fd = 3 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:506/600: open fd = 3 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:507/600: open fd = 3 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:508/600: open fd = 3 2024-03-18T23:19:19.553 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:509/600: open fd = 3 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:510/600: open fd = 3 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:511/600: open fd = 3 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.554 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:512/600: open fd = 3 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:513/600: open fd = 3 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:514/600: open fd = 3 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:515/600: open fd = 3 2024-03-18T23:19:19.555 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:516/600: open fd = 3 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:517/600: open fd = 3 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:518/600: open fd = 3 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.556 INFO:tasks.workunit.client.0.smithi132.stdout:519/600: open fd = 3 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:520/600: open fd = 3 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:521/600: open fd = 3 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:522/600: open fd = 3 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.557 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:523/600: open fd = 3 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:524/600: open fd = 3 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:525/600: open fd = 3 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.558 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:526/600: open fd = 3 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:527/600: open fd = 3 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:528/600: open fd = 3 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:529/600: open fd = 3 2024-03-18T23:19:19.559 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:530/600: open fd = 3 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:531/600: open fd = 3 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:532/600: open fd = 3 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.560 INFO:tasks.workunit.client.0.smithi132.stdout:533/600: open fd = 3 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:534/600: open fd = 3 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:535/600: open fd = 3 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:536/600: open fd = 3 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.561 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:537/600: open fd = 3 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:538/600: open fd = 3 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:539/600: open fd = 3 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:540/600: open fd = 3 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.562 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:541/600: open fd = 3 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:542/600: open fd = 3 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:543/600: open fd = 3 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.563 INFO:tasks.workunit.client.0.smithi132.stdout:544/600: open fd = 3 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:545/600: open fd = 3 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:546/600: open fd = 3 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:547/600: open fd = 3 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.564 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:548/600: open fd = 3 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:549/600: open fd = 3 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:550/600: open fd = 3 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.565 INFO:tasks.workunit.client.0.smithi132.stdout:551/600: open fd = 3 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:552/600: open fd = 3 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:553/600: open fd = 3 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:554/600: open fd = 3 2024-03-18T23:19:19.566 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:555/600: open fd = 3 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:556/600: open fd = 3 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:557/600: open fd = 3 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.567 INFO:tasks.workunit.client.0.smithi132.stdout:558/600: open fd = 3 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:559/600: open fd = 3 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:560/600: open fd = 3 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:19.568 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:21.274 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:21 smithi196 ceph-mon[67089]: pgmap v313: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:19:21.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:21 smithi144 ceph-mon[66781]: pgmap v313: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:19:21.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:21 smithi132 ceph-mon[32020]: pgmap v313: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-03-18T23:19:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 -- 172.21.15.132:0/3073872271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c068ea0 msgr2=0x7f319c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.675 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3073872271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c068ea0 0x7f319c069300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31900098c0 tx=0x7f319002bb50 comp rx=0 tx=0).stop 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 -- 172.21.15.132:0/3073872271 shutdown_connections 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3073872271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c069840 0x7f319c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3073872271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c068ea0 0x7f319c069300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3073872271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 -- 172.21.15.132:0/3073872271 >> 172.21.15.132:0/3073872271 conn(0x7f319c077570 msgr2=0x7f319c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 -- 172.21.15.132:0/3073872271 shutdown_connections 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.673+0000 7f31a12b2640 1 -- 172.21.15.132:0/3073872271 wait complete. 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 Processor -- start 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 -- start start 2024-03-18T23:19:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c069840 0x7f319c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111bf0 con 0x7f319c068ea0 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111a70 con 0x7f319c1078f0 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f31a12b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111d70 con 0x7f319c069840 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319affd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57652/0 (socket says 172.21.15.132:57652) 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319affd640 1 -- 172.21.15.132:0/3110667703 learned_addr learned my addr 172.21.15.132:0/3110667703 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319a7fc640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c069840 0x7f319c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 -- 172.21.15.132:0/3110667703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c069840 msgr2=0x7f319c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c069840 0x7f319c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 -- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 msgr2=0x7f319c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319b7fe640 1 -- 172.21.15.132:0/3110667703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c19d040 con 0x7f319c1078f0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.674+0000 7f319affd640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f319b7fe640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c19c910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f319400e940 tx=0x7f319400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f3183fff640 1 -- 172.21.15.132:0/3110667703 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400ccc0 con 0x7f319c1078f0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c19d330 con 0x7f319c1078f0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f3183fff640 1 -- 172.21.15.132:0/3110667703 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31940035f0 con 0x7f319c1078f0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f3183fff640 1 -- 172.21.15.132:0/3110667703 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400ce60 con 0x7f319c1078f0 2024-03-18T23:19:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.675+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a4bc0 con 0x7f319c1078f0 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.677+0000 7f3183fff640 1 -- 172.21.15.132:0/3110667703 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f31940037a0 con 0x7f319c1078f0 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.677+0000 7f31a3f7e740 1 Processor -- start 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 -- start start 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f3183fff640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 0x7f31840a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb2a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561bdca82be0 con 0x561bdcb285e0 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561bdca2b960 con 0x561bdcb1f810 2024-03-18T23:19:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bdca7ec80 con 0x561bdcb1eef0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319a7fc640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 0x7f31840a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f3183fff640 1 -- 172.21.15.132:0/3110667703 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f31940cf170 con 0x7f319c1078f0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb2a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb280a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:46146/0 (socket says 172.21.15.132:46146) 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:35220/0 (socket says 172.21.15.132:35220) 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319a7fc640 1 -- 172.21.15.132:0/2051754102 learned_addr learned my addr 172.21.15.132:0/2051754102 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 -- 172.21.15.132:0/2051754102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 msgr2=0x561bdcb280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 -- 172.21.15.132:0/2051754102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 msgr2=0x561bdcb2a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb2a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 -- 172.21.15.132:0/2051754102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bdca22d20 con 0x561bdcb1eef0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319b7fe640 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb2a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319a7fc640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 0x7f31840a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f319002c160 tx=0x7f3190004040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.678+0000 7f319affd640 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f318c0117f0 tx=0x7f318c011cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de9719a3a9cfca38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31817fa640 1 -- 172.21.15.132:0/2051754102 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0112f0 con 0x561bdcb1eef0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31817fa640 1 -- 172.21.15.132:0/2051754102 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c003e50 con 0x561bdcb1eef0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31817fa640 1 -- 172.21.15.132:0/2051754102 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0045c0 con 0x561bdcb1eef0 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 -- 172.21.15.132:0/2051754102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 msgr2=0x561bdcb1f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f318c0117f0 tx=0x7f318c011cb0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 -- 172.21.15.132:0/2051754102 shutdown_connections 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb2a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/2051754102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb1f2d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 -- 172.21.15.132:0/2051754102 >> 172.21.15.132:0/2051754102 conn(0x561bdcb19810 msgr2=0x561bdcb19bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 -- 172.21.15.132:0/2051754102 shutdown_connections 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.679+0000 7f31a3f7e740 1 -- 172.21.15.132:0/2051754102 wait complete. 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 -- start start 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 Processor -- start 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x561bdcb2b210 con 0x561bdcb1f810 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561bdca3edb0 con 0x561bdcb285e0 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561bdcb2b390 con 0x561bdcb1eef0 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f31a3f7e740 10 client.0 _ll_register_callbacks cb 0x561bdca43e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:35234/0 (socket says 172.21.15.132:35234) 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57678/0 (socket says 172.21.15.132:57678) 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319affd640 1 -- 172.21.15.132:0/1708281216 learned_addr learned my addr 172.21.15.132:0/1708281216 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319b7fe640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 -- 172.21.15.132:0/1708281216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 msgr2=0x561bdcb27ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 -- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 msgr2=0x561bdcb257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319a7fc640 1 -- 172.21.15.132:0/1708281216 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bdca3dc60 con 0x561bdcb1f810 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319affd640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.680+0000 7f319b7fe640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f31900057c0 tx=0x7f319002bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3975e495f00d9399 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190056060 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c003e50 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f319002c920 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f319002d500 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 10 client.16200 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561bdc797210 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c003e50 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3190051770 con 0x561bdcb1f810 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 10 client.16200 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 20 client.16200 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb383e0 0x561bdcb3a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 4 client.16200 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.681+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bdca90120 con 0x561bdcb383e0 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.682+0000 7f319affd640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb383e0 0x561bdcb3a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.682+0000 7f317eff5640 10 client.16200 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.682+0000 7f319affd640 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb383e0 0x561bdcb3a7d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5898ef2cde9a8b7c server_cookie=14203a414f20b1bb in_seq=0 out_seq=0 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f317eff5640 1 -- 172.21.15.132:0/1708281216 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561bdca90120 con 0x561bdcb383e0 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f317eff5640 10 client.16200 handle_command_reply: tid=0 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 client.16200 shutdown 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 msgr2=0x561bdcb25280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f31900057c0 tx=0x7f319002bb50 comp rx=0 tx=0).stop 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 shutdown_connections 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb383e0 0x561bdcb3a7d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb25280 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1708281216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f317eff5640 0 client.16200 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 >> 172.21.15.132:0/1708281216 conn(0x561bdcb19810 msgr2=0x561bdcb18ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 shutdown_connections 2024-03-18T23:19:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1708281216 wait complete. 2024-03-18T23:19:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.683+0000 7f31a3f7e740 20 client.16200 trim_cache size 0 max 16384 2024-03-18T23:19:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 Processor -- start 2024-03-18T23:19:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 -- start start 2024-03-18T23:19:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb33fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1afd50 con 0x561bdcb1f810 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ad620 con 0x561bdcb285e0 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.697+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ab490 con 0x561bdcb1eef0 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57690/0 (socket says 172.21.15.132:57690) 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 -- 172.21.15.132:0/604137237 learned_addr learned my addr 172.21.15.132:0/604137237 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319b7fe640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319affd640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb33fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 -- 172.21.15.132:0/604137237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 msgr2=0x561bdcb33fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb33fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 -- 172.21.15.132:0/604137237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 msgr2=0x561bdcb279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 -- 172.21.15.132:0/604137237 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bdcb1f640 con 0x561bdcb1f810 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319b7fe640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f319a7fc640 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3190000c00 tx=0x7f319002ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e209aef44f26e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f3180ff9640 1 -- 172.21.15.132:0/604137237 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190068070 con 0x561bdcb1f810 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f3180ff9640 1 -- 172.21.15.132:0/604137237 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190004130 con 0x561bdcb1f810 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.698+0000 7f3180ff9640 1 -- 172.21.15.132:0/604137237 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190056040 con 0x561bdcb1f810 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- 172.21.15.132:0/604137237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 msgr2=0x561bdcb40040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3190000c00 tx=0x7f319002ca70 comp rx=0 tx=0).stop 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- 172.21.15.132:0/604137237 shutdown_connections 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb285e0 0x561bdcb279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb1f810 0x561bdcb40040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/604137237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb33fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- 172.21.15.132:0/604137237 >> 172.21.15.132:0/604137237 conn(0x561bdcb19810 msgr2=0x561bdcb18ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- 172.21.15.132:0/604137237 shutdown_connections 2024-03-18T23:19:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- 172.21.15.132:0/604137237 wait complete. 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- start start 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 Processor -- start 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb21900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb21e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1abae0 con 0x561bdcb285e0 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1acf50 con 0x561bdcb1f810 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ac1b0 con 0x561bdcb1eef0 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.699+0000 7f31a3f7e740 10 client.0 _ll_register_callbacks cb 0x7f319002c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb21e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:35260/0 (socket says 172.21.15.132:35260) 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 -- 172.21.15.132:0/1697656620 learned_addr learned my addr 172.21.15.132:0/1697656620 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb21900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 -- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 msgr2=0x561bdcb21900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb21900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 -- 172.21.15.132:0/1697656620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 msgr2=0x561bdcb21e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb21e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 -- 172.21.15.132:0/1697656620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bdc863620 con 0x561bdcb1eef0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319b7fe640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb21e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f319affd640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f318c014040 tx=0x7f318c003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d61b5973e62241 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f317eff5640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c02e060 con 0x561bdcb1eef0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f318c011300 con 0x561bdcb1eef0 2024-03-18T23:19:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190004130 con 0x561bdcb1eef0 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f318c00fac0 con 0x561bdcb1eef0 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f31a3f7e740 10 client.26050 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561bdcb9e0a0 con 0x561bdcb1eef0 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.700+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190004130 con 0x561bdcb1eef0 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f318c018020 con 0x561bdcb1eef0 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 10 client.26050 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 20 client.26050 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb9e660 0x561bdcba0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 4 client.26050 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bdca8a470 con 0x561bdcb9e660 2024-03-18T23:19:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb9e660 0x561bdcba0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f317eff5640 10 client.26050 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.701+0000 7f319a7fc640 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb9e660 0x561bdcba0a80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7429ac256724b2bd server_cookie=6ad8c6e1507055e1 in_seq=0 out_seq=0 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f317eff5640 1 -- 172.21.15.132:0/1697656620 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bdca8a470 con 0x561bdcb9e660 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f317eff5640 10 client.26050 handle_command_reply: tid=0 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 client.26050 shutdown 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 msgr2=0x561bdcb305e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f318c014040 tx=0x7f318c003f00 comp rx=0 tx=0).stop 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 shutdown_connections 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x561bdcb9e660 0x561bdcba0a80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561bdcb285e0 0x561bdcb21e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f317eff5640 0 client.26050 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561bdcb1f810 0x561bdcb21900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 --2- 172.21.15.132:0/1697656620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561bdcb1eef0 0x561bdcb305e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 >> 172.21.15.132:0/1697656620 conn(0x561bdcb19810 msgr2=0x561bdcb18ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 shutdown_connections 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 1 -- 172.21.15.132:0/1697656620 wait complete. 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.708+0000 7f31a3f7e740 20 client.26050 trim_cache size 0 max 16384 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:19:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:19:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:19:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 msgr2=0x7f31840a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 0x7f31840a4ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f319002c160 tx=0x7f3190004040 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 msgr2=0x7f319c19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c19c910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f319400e940 tx=0x7f319400ee00 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 shutdown_connections 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f31840a2a10 0x7f31840a4ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f319c1078f0 0x7f319c19c910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c069840 0x7f319c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 --2- 172.21.15.132:0/3110667703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f319c068ea0 0x7f319c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 >> 172.21.15.132:0/3110667703 conn(0x7f319c077570 msgr2=0x7f319c0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 shutdown_connections 2024-03-18T23:19:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:21.709+0000 7f31a12b2640 1 -- 172.21.15.132:0/3110667703 wait complete. 2024-03-18T23:19:23.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:23 smithi144 ceph-mon[66781]: pgmap v314: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:23 smithi132 ceph-mon[32020]: pgmap v314: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:23.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:23 smithi196 ceph-mon[67089]: pgmap v314: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:25.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:25 smithi144 ceph-mon[66781]: pgmap v315: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:25.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:25 smithi132 ceph-mon[32020]: pgmap v315: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:25.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:25 smithi196 ceph-mon[67089]: pgmap v315: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:27.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:27 smithi144 ceph-mon[66781]: pgmap v316: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:27.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:27 smithi132 ceph-mon[32020]: pgmap v316: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:27.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:27 smithi196 ceph-mon[67089]: pgmap v316: 97 pgs: 97 active+clean; 5.5 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-03-18T23:19:28.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:28 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:28.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:28 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:28.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:28 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:19:29.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:29 smithi132 ceph-mon[32020]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:29.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:29 smithi144 ceph-mon[66781]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:29.472 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:29 smithi196 ceph-mon[67089]: pgmap v317: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:30.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:30.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:19:30.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:30 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:19:30.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:30.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.864 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:19:30.865 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:19:30.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:30 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: pgmap v318: 97 pgs: 97 active+clean; 5.6 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:30.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi196 to 174.1M 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi196 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi132 to 176.2M 2024-03-18T23:19:30.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:30 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi132 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:19:31.858 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:19:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:31.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:31 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:31.866 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:31 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:19:31.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:31 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:19:32.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:32 smithi196 ceph-mon[67089]: pgmap v319: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:33.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:32 smithi144 ceph-mon[66781]: pgmap v319: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:33.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:32 smithi132 ceph-mon[32020]: pgmap v319: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 -- 172.21.15.132:0/2295608114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de0102a90 msgr2=0x7f4de0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 --2- 172.21.15.132:0/2295608114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de0102a90 0x7f4de0102e70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009900 tx=0x7f4dd402bcd0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 -- 172.21.15.132:0/2295608114 shutdown_connections 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 --2- 172.21.15.132:0/2295608114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 0x7f4de010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 --2- 172.21.15.132:0/2295608114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de01033b0 0x7f4de010b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 --2- 172.21.15.132:0/2295608114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de0102a90 0x7f4de0102e70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.803+0000 7f4de7537640 1 -- 172.21.15.132:0/2295608114 >> 172.21.15.132:0/2295608114 conn(0x7f4de0077840 msgr2=0x7f4de0077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- 172.21.15.132:0/2295608114 shutdown_connections 2024-03-18T23:19:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- 172.21.15.132:0/2295608114 wait complete. 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 Processor -- start 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- start start 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 0x7f4de0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 0x7f4de019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4de0111bf0 con 0x7f4de01033b0 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4de0111a70 con 0x7f4de010be60 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.804+0000 7f4de7537640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4de0111d70 con 0x7f4de0102a90 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 0x7f4de0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57438/0 (socket says 172.21.15.132:57438) 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 -- 172.21.15.132:0/217522754 learned_addr learned my addr 172.21.15.132:0/217522754 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 0x7f4de019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 -- 172.21.15.132:0/217522754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 msgr2=0x7f4de0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 0x7f4de0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 -- 172.21.15.132:0/217522754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 msgr2=0x7f4de019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 0x7f4de019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 -- 172.21.15.132:0/217522754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de019d020 con 0x7f4de01033b0 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de52d3640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 0x7f4de0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00e9f0 tx=0x7f4ddc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4dd27fc640 1 -- 172.21.15.132:0/217522754 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00ccc0 con 0x7f4de01033b0 2024-03-18T23:19:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de019d310 con 0x7f4de01033b0 2024-03-18T23:19:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4dd27fc640 1 -- 172.21.15.132:0/217522754 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ddc00ce60 con 0x7f4de01033b0 2024-03-18T23:19:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4dd27fc640 1 -- 172.21.15.132:0/217522754 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc010690 con 0x7f4de01033b0 2024-03-18T23:19:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.805+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de01a4ba0 con 0x7f4de01033b0 2024-03-18T23:19:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.808+0000 7f4dd27fc640 1 -- 172.21.15.132:0/217522754 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f4ddc0046c0 con 0x7f4de01033b0 2024-03-18T23:19:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.808+0000 7f4dea1f8740 1 Processor -- start 2024-03-18T23:19:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.808+0000 7f4dea1f8740 1 -- start start 2024-03-18T23:19:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dd27fc640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 0x7f4dcc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 0x55e116b4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b4d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e116aa5be0 con 0x55e116b41ef0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e116a4e960 con 0x55e116b4b5e0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e116aa1c80 con 0x55e116b42810 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de5ad4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de52d3640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 0x7f4dcc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de5ad4640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57448/0 (socket says 172.21.15.132:57448) 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de5ad4640 1 -- 172.21.15.132:0/2208833184 learned_addr learned my addr 172.21.15.132:0/2208833184 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 0x55e116b4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b4d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4dd27fc640 1 -- 172.21.15.132:0/217522754 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f4ddc0cf220 con 0x7f4de01033b0 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 -- 172.21.15.132:0/2208833184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 msgr2=0x55e116b4d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b4d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 -- 172.21.15.132:0/2208833184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 msgr2=0x55e116b422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 -- 172.21.15.132:0/2208833184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e116a45d20 con 0x55e116b42810 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de52d3640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 0x7f4dcc0a4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd402efd0 tx=0x7f4dd402cf90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.809+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 0x55e116b4b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0cc970 tx=0x7f4ddc004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b7ad1430acad70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dcb7fe640 1 -- 172.21.15.132:0/2208833184 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc099e40 con 0x55e116b42810 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dcb7fe640 1 -- 172.21.15.132:0/2208833184 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ddc09bf80 con 0x55e116b42810 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dcb7fe640 1 -- 172.21.15.132:0/2208833184 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc098ed0 con 0x55e116b42810 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 -- 172.21.15.132:0/2208833184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 msgr2=0x55e116b4b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 0x55e116b4b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0cc970 tx=0x7f4ddc004b00 comp rx=0 tx=0).stop 2024-03-18T23:19:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 -- 172.21.15.132:0/2208833184 shutdown_connections 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b4d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b42810 0x55e116b4b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/2208833184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 -- 172.21.15.132:0/2208833184 >> 172.21.15.132:0/2208833184 conn(0x55e116b3c810 msgr2=0x55e116b3cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 -- 172.21.15.132:0/2208833184 shutdown_connections 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.810+0000 7f4dea1f8740 1 -- 172.21.15.132:0/2208833184 wait complete. 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 -- start start 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 Processor -- start 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b4b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55e116b4e150 con 0x55e116b42810 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e116a61db0 con 0x55e116b4b5e0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e116b4e2d0 con 0x55e116b41ef0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4dea1f8740 10 client.0 _ll_register_callbacks cb 0x55e116a66e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de4ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b4b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de5ad4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de5ad4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:33232/0 (socket says 172.21.15.132:33232) 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de5ad4640 1 -- 172.21.15.132:0/344006411 learned_addr learned my addr 172.21.15.132:0/344006411 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.811+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b4b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:39538/0 (socket says 172.21.15.132:39538) 2024-03-18T23:19:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 -- 172.21.15.132:0/344006411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 msgr2=0x55e116b4b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b4b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 -- 172.21.15.132:0/344006411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 msgr2=0x55e116b483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 -- 172.21.15.132:0/344006411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e116a60c60 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80099a0 tx=0x7f4dd8009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d7b1ee3068dfb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8003970 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ddc09bf80 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f4dd8003b10 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4dd8003d10 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dea1f8740 10 client.26063 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e1167ba210 con 0x55e116b4b5e0 2024-03-18T23:19:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.812+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ddc09bf80 con 0x55e116b4b5e0 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4dd80105d0 con 0x55e116b4b5e0 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 10 client.26063 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 20 client.26063 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116b5b470 0x55e116b5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 4 client.26063 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e116ab3120 con 0x55e116b5b470 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4de52d3640 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116b5b470 0x55e116b5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4dc8ff9640 10 client.26063 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.813+0000 7f4de52d3640 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116b5b470 0x55e116b5d860 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5818128fc8766e96 server_cookie=30b9c9d9a7ff5aa6 in_seq=0 out_seq=0 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/344006411 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e116ab3120 con 0x55e116b5b470 2024-03-18T23:19:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dc8ff9640 10 client.26063 handle_command_reply: tid=0 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 client.26063 shutdown 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 msgr2=0x55e116b48ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80099a0 tx=0x7f4dd8009ea0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 shutdown_connections 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116b5b470 0x55e116b5d860 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116b48ae0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dc8ff9640 0 client.26063 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/344006411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b4b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.814+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 >> 172.21.15.132:0/344006411 conn(0x55e116b3c810 msgr2=0x55e116b3bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.815+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 shutdown_connections 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.815+0000 7f4dea1f8740 1 -- 172.21.15.132:0/344006411 wait complete. 2024-03-18T23:19:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.815+0000 7f4dea1f8740 20 client.26063 trim_cache size 0 max 16384 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.829+0000 7f4dea1f8740 1 Processor -- start 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.829+0000 7f4dea1f8740 1 -- start start 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b42810 0x55e116b63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b4b5e0 0x55e116b4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01afdc0 con 0x55e116b41ef0 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01ad690 con 0x55e116b42810 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01ab500 con 0x55e116b4b5e0 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57458/0 (socket says 172.21.15.132:57458) 2024-03-18T23:19:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 -- 172.21.15.132:0/569527054 learned_addr learned my addr 172.21.15.132:0/569527054 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b42810 0x55e116b63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b4b5e0 0x55e116b4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 -- 172.21.15.132:0/569527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b4b5e0 msgr2=0x55e116b4aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b4b5e0 0x55e116b4aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 -- 172.21.15.132:0/569527054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b42810 msgr2=0x55e116b63130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b42810 0x55e116b63130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 -- 172.21.15.132:0/569527054 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e116886620 con 0x55e116b41ef0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4de52d3640 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009230 tx=0x7f4dd403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56720f635b07205 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dcaffd640 1 -- 172.21.15.132:0/569527054 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4059070 con 0x55e116b41ef0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dcaffd640 1 -- 172.21.15.132:0/569527054 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd4055d10 con 0x55e116b41ef0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.830+0000 7f4dcaffd640 1 -- 172.21.15.132:0/569527054 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4005190 con 0x55e116b41ef0 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- 172.21.15.132:0/569527054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 msgr2=0x55e116b56ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009230 tx=0x7f4dd403f000 comp rx=0 tx=0).stop 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- 172.21.15.132:0/569527054 shutdown_connections 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b4b5e0 0x55e116b4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b42810 0x55e116b63130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/569527054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b41ef0 0x55e116b56ef0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- 172.21.15.132:0/569527054 >> 172.21.15.132:0/569527054 conn(0x55e116b3c810 msgr2=0x55e116b3bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- 172.21.15.132:0/569527054 shutdown_connections 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- 172.21.15.132:0/569527054 wait complete. 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 -- start start 2024-03-18T23:19:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.831+0000 7f4dea1f8740 1 Processor -- start 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116bc0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01abb50 con 0x55e116b42810 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01acfc0 con 0x55e116b4b5e0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01ac220 con 0x55e116b41ef0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dea1f8740 10 client.0 _ll_register_callbacks cb 0x7f4dd8013c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de52d3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de5ad4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116bc0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:57472/0 (socket says 172.21.15.132:57472) 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 -- 172.21.15.132:0/3403992840 learned_addr learned my addr 172.21.15.132:0/3403992840 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 -- 172.21.15.132:0/3403992840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 msgr2=0x55e116b45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b45290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 -- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 msgr2=0x55e116bc0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116bc0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 -- 172.21.15.132:0/3403992840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e116b46060 con 0x55e116b42810 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de52d3640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de5ad4640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116bc0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4de4ad2640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc09b580 tx=0x7f4ddc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb18ff215fcab14a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.832+0000 7f4dc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0990b0 con 0x55e116b42810 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f4ddc0a3b90 con 0x55e116b42810 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd4055d10 con 0x55e116b42810 2024-03-18T23:19:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ddc099760 con 0x55e116b42810 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 10 client.16224 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e116bc1d80 con 0x55e116b42810 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd4055d10 con 0x55e116b42810 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4ddc09cbb0 con 0x55e116b42810 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 10 client.16224 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 20 client.16224 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116bc2180 0x55e116bc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 4 client.16224 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e116aad470 con 0x55e116bc2180 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.833+0000 7f4de52d3640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116bc2180 0x55e116bc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.834+0000 7f4dc8ff9640 10 client.16224 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.834+0000 7f4de52d3640 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116bc2180 0x55e116bc4570 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6811283a5c8d01c1 server_cookie=811dab0101174146 in_seq=0 out_seq=0 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dc8ff9640 1 -- 172.21.15.132:0/3403992840 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e116aad470 con 0x55e116bc2180 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dc8ff9640 10 client.16224 handle_command_reply: tid=0 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 client.16224 shutdown 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 msgr2=0x55e116b457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc09b580 tx=0x7f4ddc005f10 comp rx=0 tx=0).stop 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 shutdown_connections 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x55e116bc2180 0x55e116bc4570 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e116b4b5e0 0x55e116bc0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e116b42810 0x55e116b457d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dc8ff9640 0 client.16224 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 --2- 172.21.15.132:0/3403992840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e116b41ef0 0x55e116b45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 >> 172.21.15.132:0/3403992840 conn(0x55e116b3c810 msgr2=0x55e116b43020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 shutdown_connections 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 1 -- 172.21.15.132:0/3403992840 wait complete. 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.841+0000 7f4dea1f8740 20 client.16224 trim_cache size 0 max 16384 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:19:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:19:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 msgr2=0x7f4dcc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 0x7f4dcc0a4ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4dd402efd0 tx=0x7f4dd402cf90 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 msgr2=0x7f4de0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00e9f0 tx=0x7f4ddc00edf0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 shutdown_connections 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f4dcc0a2a10 0x7f4dcc0a4ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4de010be60 0x7f4de019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01033b0 0x7f4de0198570 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 --2- 172.21.15.132:0/217522754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4de0102a90 0x7f4de0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 >> 172.21.15.132:0/217522754 conn(0x7f4de0077840 msgr2=0x7f4de00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 shutdown_connections 2024-03-18T23:19:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:34.842+0000 7f4de7537640 1 -- 172.21.15.132:0/217522754 wait complete. 2024-03-18T23:19:34.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:34 smithi196 ceph-mon[67089]: pgmap v320: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:35.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:34 smithi144 ceph-mon[66781]: pgmap v320: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:35.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:34 smithi132 ceph-mon[32020]: pgmap v320: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:36.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:36 smithi196 ceph-mon[67089]: pgmap v321: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:37.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:36 smithi144 ceph-mon[66781]: pgmap v321: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:36 smithi132 ceph-mon[32020]: pgmap v321: 97 pgs: 97 active+clean; 5.6 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:38.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:38 smithi196 ceph-mon[67089]: pgmap v322: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:19:39.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:38 smithi144 ceph-mon[66781]: pgmap v322: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:19:39.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:38 smithi132 ceph-mon[32020]: pgmap v322: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-03-18T23:19:40.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:40 smithi196 ceph-mon[67089]: pgmap v323: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:41.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:40 smithi144 ceph-mon[66781]: pgmap v323: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:41.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:40 smithi132 ceph-mon[32020]: pgmap v323: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:42.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:42 smithi196 ceph-mon[67089]: pgmap v324: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:43.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:42 smithi144 ceph-mon[66781]: pgmap v324: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:43.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:42 smithi132 ceph-mon[32020]: pgmap v324: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:44.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:44 smithi196 ceph-mon[67089]: pgmap v325: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:45.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:44 smithi144 ceph-mon[66781]: pgmap v325: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:45.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:44 smithi132 ceph-mon[32020]: pgmap v325: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:46.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:46 smithi196 ceph-mon[67089]: pgmap v326: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:47.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:46 smithi144 ceph-mon[66781]: pgmap v326: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:47.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:46 smithi132 ceph-mon[32020]: pgmap v326: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.936+0000 7f5c9d836640 1 -- 172.21.15.132:0/3378205956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806ab20 msgr2=0x7f5c9810e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.936+0000 7f5c9d836640 1 --2- 172.21.15.132:0/3378205956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806ab20 0x7f5c9810e6a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009c40 tx=0x7f5c9002bed0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 -- 172.21.15.132:0/3378205956 shutdown_connections 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 --2- 172.21.15.132:0/3378205956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806ab20 0x7f5c9810e6a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 --2- 172.21.15.132:0/3378205956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c9806a160 0x7f5c9806a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 --2- 172.21.15.132:0/3378205956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c98107890 0x7f5c98107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 -- 172.21.15.132:0/3378205956 >> 172.21.15.132:0/3378205956 conn(0x7f5c980778a0 msgr2=0x7f5c98077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 -- 172.21.15.132:0/3378205956 shutdown_connections 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 -- 172.21.15.132:0/3378205956 wait complete. 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 Processor -- start 2024-03-18T23:19:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.937+0000 7f5c9d836640 1 -- start start 2024-03-18T23:19:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 0x7f5c98198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5c98111be0 con 0x7f5c98107890 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c98111a60 con 0x7f5c9806a160 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c98111d60 con 0x7f5c9806ab20 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46544/0 (socket says 172.21.15.132:46544) 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 0x7f5c98198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:45936/0 (socket says 172.21.15.132:45936) 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 -- 172.21.15.132:0/2675948736 learned_addr learned my addr 172.21.15.132:0/2675948736 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 -- 172.21.15.132:0/2675948736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 msgr2=0x7f5c98198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 0x7f5c98198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 -- 172.21.15.132:0/2675948736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 msgr2=0x7f5c98197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 -- 172.21.15.132:0/2675948736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9819cf80 con 0x7f5c98107890 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 0x7f5c98198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c977fe640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c90033420 tx=0x7f5c900337f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9c834640 1 -- 172.21.15.132:0/2675948736 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90043020 con 0x7f5c98107890 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.938+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9819d210 con 0x7f5c98107890 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.939+0000 7f5c9c834640 1 -- 172.21.15.132:0/2675948736 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c90003b80 con 0x7f5c98107890 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.939+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c981a4b00 con 0x7f5c98107890 2024-03-18T23:19:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.939+0000 7f5c9c834640 1 -- 172.21.15.132:0/2675948736 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9003e7d0 con 0x7f5c98107890 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5c9c834640 1 -- 172.21.15.132:0/2675948736 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f5c90046020 con 0x7f5c98107890 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5ca0426740 1 Processor -- start 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5c9c834640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 0x7f5c840a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5ca0426740 1 -- start start 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 0x7f5c840a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5c9c834640 1 -- 172.21.15.132:0/2675948736 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f5c900f1610 con 0x7f5c98107890 2024-03-18T23:19:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.941+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c14810 0x563e92c1d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 0x563e92c1f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5ca0426740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563e92b77be0 con 0x563e92c1d5e0 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5ca0426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e92b20960 con 0x563e92c13ef0 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5ca0426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e92b73c80 con 0x563e92c14810 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 0x563e92c1f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:45944/0 (socket says 172.21.15.132:45944) 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 -- 172.21.15.132:0/3574124463 learned_addr learned my addr 172.21.15.132:0/3574124463 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c14810 0x563e92c1d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 -- 172.21.15.132:0/3574124463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c14810 msgr2=0x563e92c1d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c14810 0x563e92c1d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 -- 172.21.15.132:0/3574124463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 msgr2=0x563e92c1f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 0x563e92c1f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 -- 172.21.15.132:0/3574124463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e92b17d20 con 0x563e92c13ef0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c977fe640 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 0x563e92c1f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 0x7f5c840a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c980fd890 tx=0x7f5c80011ce0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c88011540 tx=0x7f5c88011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7f09f248d3cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c8dffb640 1 -- 172.21.15.132:0/3574124463 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800f5c0 con 0x563e92c13ef0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c8dffb640 1 -- 172.21.15.132:0/3574124463 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8800fc40 con 0x563e92c13ef0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.942+0000 7f5c8dffb640 1 -- 172.21.15.132:0/3574124463 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88019cc0 con 0x563e92c13ef0 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- 172.21.15.132:0/3574124463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 msgr2=0x563e92c142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c88011540 tx=0x7f5c88011a00 comp rx=0 tx=0).stop 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- 172.21.15.132:0/3574124463 shutdown_connections 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c1d5e0 0x563e92c1f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c14810 0x563e92c1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 --2- 172.21.15.132:0/3574124463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c142d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- 172.21.15.132:0/3574124463 >> 172.21.15.132:0/3574124463 conn(0x563e92c0e810 msgr2=0x563e92c0ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- 172.21.15.132:0/3574124463 shutdown_connections 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- 172.21.15.132:0/3574124463 wait complete. 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.943+0000 7f5ca0426740 1 -- start start 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 Processor -- start 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c1a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x563e92c20210 con 0x563e92c14810 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e92b33db0 con 0x563e92c13ef0 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e92c20390 con 0x563e92c1d5e0 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5ca0426740 10 client.0 _ll_register_callbacks cb 0x563e92b38e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.951 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c1a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:45954/0 (socket says 172.21.15.132:45954) 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 -- 172.21.15.132:0/2552700670 learned_addr learned my addr 172.21.15.132:0/2552700670 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c977fe640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 -- 172.21.15.132:0/2552700670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 msgr2=0x563e92c1a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 -- 172.21.15.132:0/2552700670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 msgr2=0x563e92c1a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c1a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c96ffd640 1 -- 172.21.15.132:0/2552700670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e92b32c60 con 0x563e92c13ef0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.944+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009350 tx=0x7f5c8800fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0e57271cc08281 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88026020 con 0x563e92c13ef0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8800fc40 con 0x563e92c13ef0 2024-03-18T23:19:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f5c88019bf0 con 0x563e92c13ef0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c88031050 con 0x563e92c13ef0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 10 client.26081 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563e9288c210 con 0x563e92c13ef0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8800fc40 con 0x563e92c13ef0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5c880035f0 con 0x563e92c13ef0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 10 client.26081 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 20 client.26081 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c2d3f0 0x563e92c2f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 4 client.26081 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.945+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e92b85120 con 0x563e92c2d3f0 2024-03-18T23:19:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.946+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c2d3f0 0x563e92c2f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.946+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c2d3f0 0x563e92c2f7e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08bba9ac8e4a836 server_cookie=72d403d6af419d88 in_seq=0 out_seq=0 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.946+0000 7f5c76ffd640 10 client.26081 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.946+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2552700670 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563e92b85120 con 0x563e92c2d3f0 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.946+0000 7f5c76ffd640 10 client.26081 handle_command_reply: tid=0 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 client.26081 shutdown 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 msgr2=0x563e92c1cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009350 tx=0x7f5c8800fdf0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 shutdown_connections 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c2d3f0 0x563e92c2f7e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5c76ffd640 0 client.26081 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2552700670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c1cf70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 >> 172.21.15.132:0/2552700670 conn(0x563e92c0e810 msgr2=0x563e92c0dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 shutdown_connections 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 1 -- 172.21.15.132:0/2552700670 wait complete. 2024-03-18T23:19:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.947+0000 7f5ca0426740 20 client.26081 trim_cache size 0 max 16384 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.961+0000 7f5ca0426740 1 Processor -- start 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 -- start start 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c28fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981afd40 con 0x563e92c14810 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981ad610 con 0x563e92c13ef0 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5ca0426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981ab480 con 0x563e92c1d5e0 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46570/0 (socket says 172.21.15.132:46570) 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/1956719776 learned_addr learned my addr 172.21.15.132:0/1956719776 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c28fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c977fe640 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/1956719776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 msgr2=0x563e92c1ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/1956719776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 msgr2=0x563e92c28fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c28fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/1956719776 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e92958620 con 0x563e92c14810 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009530 tx=0x7f5c80020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77fc8cc6eb4809f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/1956719776 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8001a040 con 0x563e92c14810 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/1956719776 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80004020 con 0x563e92c14810 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.962+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/1956719776 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80005520 con 0x563e92c14810 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- 172.21.15.132:0/1956719776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 msgr2=0x563e92c34fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009530 tx=0x7f5c80020000 comp rx=0 tx=0).stop 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- 172.21.15.132:0/1956719776 shutdown_connections 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c1ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c34fb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 --2- 172.21.15.132:0/1956719776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c28fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- 172.21.15.132:0/1956719776 >> 172.21.15.132:0/1956719776 conn(0x563e92c0e810 msgr2=0x563e92c0dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- 172.21.15.132:0/1956719776 shutdown_connections 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- 172.21.15.132:0/1956719776 wait complete. 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.963+0000 7f5ca0426740 1 -- start start 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 Processor -- start 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c21000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c21a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981abad0 con 0x563e92c14810 2024-03-18T23:19:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981acf40 con 0x563e92c13ef0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c981ac1a0 con 0x563e92c1d5e0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5ca0426740 10 client.0 _ll_register_callbacks cb 0x7f5c90005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:46574/0 (socket says 172.21.15.132:46574) 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/2649608337 learned_addr learned my addr 172.21.15.132:0/2649608337 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c21000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c977fe640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/2649608337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 msgr2=0x563e92c21a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c21a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 msgr2=0x563e92c21000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c21000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 -- 172.21.15.132:0/2649608337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e92c92710 con 0x563e92c14810 2024-03-18T23:19:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c21000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c8e7fc640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000ffd0 tx=0x7f5c80004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4ed0372159f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.964+0000 7f5c76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8001a030 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mon.0 v2:172.21.15.132:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f5c80004440 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c80004020 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c80043050 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 10 client.16242 fetch_fsmap learned FSMap version 24 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563e92c93a00 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mon.0 v2:172.21.15.132:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80004020 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mon.0 v2:172.21.15.132:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5c80034070 con 0x563e92c14810 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 10 client.16242 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T23:19:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 20 client.16242 populate_metadata read hostname 'smithi132' 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c94140 0x563e92c96560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 4 client.16242 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 --> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e92b7f470 con 0x563e92c94140 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.965+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c94140 0x563e92c96560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.966+0000 7f5c96ffd640 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c94140 0x563e92c96560 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70eabd7bc2d2f5ea server_cookie=bc204f0169c76918 in_seq=0 out_seq=0 2024-03-18T23:19:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.966+0000 7f5c76ffd640 10 client.16242 ms_handle_connect on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.972+0000 7f5c76ffd640 1 -- 172.21.15.132:0/2649608337 <== mds.0 v2:172.21.15.144:6838/837888145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e92b7f470 con 0x563e92c94140 2024-03-18T23:19:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.972+0000 7f5c76ffd640 10 client.16242 handle_command_reply: tid=0 2024-03-18T23:19:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.972+0000 7f5ca0426740 1 client.16242 shutdown 2024-03-18T23:19:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 msgr2=0x563e92c21540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000ffd0 tx=0x7f5c80004250 comp rx=0 tx=0).stop 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 shutdown_connections 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:6838/837888145,v1:172.21.15.144:6839/837888145] conn(0x563e92c94140 0x563e92c96560 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e92c1d5e0 0x563e92c21a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563e92c14810 0x563e92c21540 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c76ffd640 0 client.16242 ms_handle_reset on v2:172.21.15.144:6838/837888145 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 --2- 172.21.15.132:0/2649608337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e92c13ef0 0x563e92c21000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 >> 172.21.15.132:0/2649608337 conn(0x563e92c0e810 msgr2=0x563e92c14ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 shutdown_connections 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 1 -- 172.21.15.132:0/2649608337 wait complete. 2024-03-18T23:19:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5ca0426740 20 client.16242 trim_cache size 0 max 16384 2024-03-18T23:19:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:{ 2024-03-18T23:19:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "message": "", 2024-03-18T23:19:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout: "return_code": 0 2024-03-18T23:19:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stdout:} 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 msgr2=0x7f5c840a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 0x7f5c840a4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c980fd890 tx=0x7f5c80011ce0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 msgr2=0x7f5c9819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c90033420 tx=0x7f5c900337f0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 shutdown_connections 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f5c840a2a10 0x7f5c840a4ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c98107890 0x7f5c9819c850 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c9806ab20 0x7f5c98198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 --2- 172.21.15.132:0/2675948736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9806a160 0x7f5c98197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.973+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 >> 172.21.15.132:0/2675948736 conn(0x7f5c980778a0 msgr2=0x7f5c980fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:19:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.974+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 shutdown_connections 2024-03-18T23:19:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi132.front.sepia.ceph.com.smithi132.stderr:2024-03-18T23:19:47.974+0000 7f5c9d836640 1 -- 172.21.15.132:0/2675948736 wait complete. 2024-03-18T23:19:48.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:48 smithi196 ceph-mon[67089]: pgmap v327: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:19:49.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:48 smithi144 ceph-mon[66781]: pgmap v327: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:19:49.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:48 smithi132 ceph-mon[32020]: pgmap v327: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-03-18T23:19:50.972 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:50 smithi196 ceph-mon[67089]: pgmap v328: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:19:51.107 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:50 smithi144 ceph-mon[66781]: pgmap v328: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:19:51.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:50 smithi132 ceph-mon[32020]: pgmap v328: 97 pgs: 97 active+clean; 5.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-03-18T23:19:52.973 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:52 smithi196 ceph-mon[67089]: pgmap v329: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:53.108 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:52 smithi144 ceph-mon[66781]: pgmap v329: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:53.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:52 smithi132 ceph-mon[32020]: pgmap v329: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-03-18T23:19:55.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:54 smithi196 ceph-mon[67089]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:55.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:54 smithi144 ceph-mon[66781]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:55.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:54 smithi132 ceph-mon[32020]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:57.159 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:56 smithi144 ceph-mon[66781]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:57.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:56 smithi196 ceph-mon[67089]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:57.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:56 smithi132 ceph-mon[32020]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:19:59.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:19:58 smithi196 ceph-mon[67089]: pgmap v332: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:59.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:19:58 smithi144 ceph-mon[66781]: pgmap v332: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:59.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:19:58 smithi132 ceph-mon[32020]: pgmap v332: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-03-18T23:19:59.783 INFO:tasks.workunit.client.0.smithi132.stdout:561/600: open fd = 3 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:562/600: open fd = 3 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:563/600: open fd = 3 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.784 INFO:tasks.workunit.client.0.smithi132.stdout:564/600: open fd = 3 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:565/600: open fd = 3 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:566/600: open fd = 3 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:567/600: open fd = 3 2024-03-18T23:19:59.785 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:568/600: open fd = 3 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:569/600: open fd = 3 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:570/600: open fd = 3 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.786 INFO:tasks.workunit.client.0.smithi132.stdout:571/600: open fd = 3 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:572/600: open fd = 3 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:573/600: open fd = 3 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:574/600: open fd = 3 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.787 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:575/600: open fd = 3 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:576/600: open fd = 3 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:577/600: open fd = 3 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:578/600: open fd = 3 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.788 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:579/600: open fd = 3 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:580/600: open fd = 3 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:581/600: open fd = 3 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:582/600: open fd = 3 2024-03-18T23:19:59.789 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:583/600: open fd = 3 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:584/600: open fd = 3 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:585/600: open fd = 3 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.790 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:586/600: open fd = 3 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:587/600: open fd = 3 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:588/600: open fd = 3 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:589/600: open fd = 3 2024-03-18T23:19:59.791 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:590/600: open fd = 3 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:591/600: open fd = 3 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:592/600: open fd = 3 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.792 INFO:tasks.workunit.client.0.smithi132.stdout:593/600: open fd = 3 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:594/600: open fd = 3 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:595/600: open fd = 3 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:596/600: open fd = 3 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.793 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:597/600: open fd = 3 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:598/600: open fd = 3 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:599/600: open fd = 3 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.794 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stdout:600/600: open fd = 3 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stdout:write ret = 32 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stdout:pread ret = 64 2024-03-18T23:19:59.795 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-03-18T23:19:59.796 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-03-18T23:19:59.797 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-18T23:19:59.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-18T23:19:59.851 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-03-18T23:19:59.851 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-18T23:20:00.131 DEBUG:teuthology.parallel:result is None 2024-03-18T23:20:00.132 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T23:20:00.168 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T23:20:00.168 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-18T23:20:00.180 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:20:00.216 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-18T23:20:00.226 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-18T23:20:00.226 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T23:20:00.226 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:20:00.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-18T23:20:00.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-18T23:20:00.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 -- 172.21.15.132:0/1477063379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f8101330 msgr2=0x7f79f810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1477063379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f8101330 0x7f79f810d550 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009ca0 tx=0x7f79f002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 -- 172.21.15.132:0/1477063379 shutdown_connections 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1477063379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f8101330 0x7f79f810d550 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1477063379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8100970 0x7f79f8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1477063379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f81056c0 0x7f79f8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 -- 172.21.15.132:0/1477063379 >> 172.21.15.132:0/1477063379 conn(0x7f79f8077570 msgr2=0x7f79f8077980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:00.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 -- 172.21.15.132:0/1477063379 shutdown_connections 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 -- 172.21.15.132:0/1477063379 wait complete. 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.508+0000 7f79fdc81640 1 Processor -- start 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 -- start start 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 0x7f79f8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110be0 con 0x7f79f81056c0 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110a60 con 0x7f79f8100970 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79fdc81640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8110d60 con 0x7f79f8101330 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 0x7f79f8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:50592/0 (socket says 172.21.15.132:50592) 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f77fe640 1 -- 172.21.15.132:0/1891007170 learned_addr learned my addr 172.21.15.132:0/1891007170 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43016/0 (socket says 172.21.15.132:43016) 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 -- 172.21.15.132:0/1891007170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 msgr2=0x7f79f8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 0x7f79f8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 -- 172.21.15.132:0/1891007170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 msgr2=0x7f79f8198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 -- 172.21.15.132:0/1891007170 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f81a44c0 con 0x7f79f81056c0 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f77fe640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f6ffd640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 0x7f79f8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.509+0000 7f79f7fff640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f79f002c700 tx=0x7f79f0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.510+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0042020 con 0x7f79f81056c0 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.510+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f0034640 con 0x7f79f81056c0 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.510+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f81a4690 con 0x7f79f81056c0 2024-03-18T23:20:00.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.510+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0039040 con 0x7f79f81056c0 2024-03-18T23:20:00.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.510+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f81a4b50 con 0x7f79f81056c0 2024-03-18T23:20:00.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.511+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f804f580 con 0x7f79f81056c0 2024-03-18T23:20:00.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.512+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f79f0045020 con 0x7f79f81056c0 2024-03-18T23:20:00.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.512+0000 7f79f4ff9640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 0x7f79e00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.512+0000 7f79f77fe640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 0x7f79e00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.512+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f79f00f0cc0 con 0x7f79f81056c0 2024-03-18T23:20:00.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.513+0000 7f79f77fe640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 0x7f79e00a4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011440 tx=0x7f79e8005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:00.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.516+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f79f004c020 con 0x7f79f81056c0 2024-03-18T23:20:00.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.711+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f79f8108c80 con 0x7f79e00a2a10 2024-03-18T23:20:00.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.718+0000 7f79f4ff9640 1 -- 172.21.15.132:0/1891007170 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f79e800f480 con 0x7f79e00a2a10 2024-03-18T23:20:00.718 INFO:teuthology.orchestra.run.smithi132.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T23:09:16", "first": "2024-03-18T23:10:00", "last": "2024-03-18T23:20:00", "last_pruned": "2024-03-18T23:20:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-03-18T23:20:00.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.720+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 msgr2=0x7f79e00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.720+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 0x7f79e00a4ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011440 tx=0x7f79e8005f90 comp rx=0 tx=0).stop 2024-03-18T23:20:00.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.720+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 msgr2=0x7f79f81a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.720+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f79f002c700 tx=0x7f79f0032f20 comp rx=0 tx=0).stop 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.720+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 shutdown_connections 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f79e00a2a10 0x7f79e00a4ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f81056c0 0x7f79f81a3f80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79f8101330 0x7f79f8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 --2- 172.21.15.132:0/1891007170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79f8100970 0x7f79f8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 >> 172.21.15.132:0/1891007170 conn(0x7f79f8077570 msgr2=0x7f79f80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 shutdown_connections 2024-03-18T23:20:00.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.721+0000 7f79fdc81640 1 -- 172.21.15.132:0/1891007170 wait complete. 2024-03-18T23:20:00.731 DEBUG:teuthology.orchestra.run.smithi132:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T23:20:00.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.835+0000 7ff1a4f53640 1 -- 172.21.15.132:0/4244494739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a020 msgr2=0x7ff1a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.835+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/4244494739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a020 0x7ff1a006a480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff18802bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:00.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 -- 172.21.15.132:0/4244494739 shutdown_connections 2024-03-18T23:20:00.835 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/4244494739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a006a9c0 0x7ff1a010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/4244494739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a020 0x7ff1a006a480 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/4244494739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a0107890 0x7ff1a0107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 -- 172.21.15.132:0/4244494739 >> 172.21.15.132:0/4244494739 conn(0x7ff1a00774f0 msgr2=0x7ff1a0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:00.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 -- 172.21.15.132:0/4244494739 shutdown_connections 2024-03-18T23:20:00.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 -- 172.21.15.132:0/4244494739 wait complete. 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.836+0000 7ff1a4f53640 1 Processor -- start 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 -- start start 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 0x7ff1a01a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 0x7ff1a010d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.838 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111bb0 con 0x7ff1a006a020 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111a30 con 0x7ff1a006a9c0 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1a4f53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111d30 con 0x7ff1a0107890 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43036/0 (socket says 172.21.15.132:43036) 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 -- 172.21.15.132:0/3185015960 learned_addr learned my addr 172.21.15.132:0/3185015960 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff197fff640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 0x7ff1a01a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19ed9d640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 0x7ff1a010d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 -- 172.21.15.132:0/3185015960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 msgr2=0x7ff1a010d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 0x7ff1a010d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 -- 172.21.15.132:0/3185015960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 msgr2=0x7ff1a01a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 0x7ff1a01a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 -- 172.21.15.132:0/3185015960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a010ded0 con 0x7ff1a006a020 2024-03-18T23:20:00.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19ed9d640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 0x7ff1a010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff197fff640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 0x7ff1a01a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff19e59c640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff19000ea60 tx=0x7ff19000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.837+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19000cde0 con 0x7ff1a006a020 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.838+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a010e1c0 con 0x7ff1a006a020 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.838+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1900035f0 con 0x7ff1a006a020 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.838+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190010670 con 0x7ff1a006a020 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.838+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01ab080 con 0x7ff1a006a020 2024-03-18T23:20:00.840 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.838+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a0102f20 con 0x7ff1a006a020 2024-03-18T23:20:00.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.840+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7ff1900037a0 con 0x7ff1a006a020 2024-03-18T23:20:00.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.840+0000 7ff1977fe640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 0x7ff18c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:00.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.840+0000 7ff197fff640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 0x7ff18c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:00.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.840+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7ff1900cf0c0 con 0x7ff1a006a020 2024-03-18T23:20:00.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.841+0000 7ff197fff640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 0x7ff18c0a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188005b50 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:00.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:00.845+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff19009bec0 con 0x7ff1a006a020 2024-03-18T23:20:01.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.047+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 --> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff1a00ff940 con 0x7ff18c0a2a10 2024-03-18T23:20:01.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.054+0000 7ff1977fe640 1 -- 172.21.15.132:0/3185015960 <== mgr.15294 v2:172.21.15.132:6800/4165175733 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff1a00ff940 con 0x7ff18c0a2a10 2024-03-18T23:20:01.053 INFO:teuthology.orchestra.run.smithi132.stdout:/ 1m 6m3h 2024-03-18T23:20:01.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 msgr2=0x7ff18c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 0x7ff18c0a4ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff188006010 tx=0x7ff188005b50 comp rx=0 tx=0).stop 2024-03-18T23:20:01.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 msgr2=0x7ff1a01a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff19000ea60 tx=0x7ff19000ef20 comp rx=0 tx=0).stop 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 shutdown_connections 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7ff18c0a2a10 0x7ff18c0a4ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1a0107890 0x7ff1a010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a006a9c0 0x7ff1a01a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 --2- 172.21.15.132:0/3185015960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1a006a020 0x7ff1a01a4230 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 >> 172.21.15.132:0/3185015960 conn(0x7ff1a00774f0 msgr2=0x7ff1a00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 shutdown_connections 2024-03-18T23:20:01.056 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.056+0000 7ff1a4f53640 1 -- 172.21.15.132:0/3185015960 wait complete. 2024-03-18T23:20:01.065 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:00 smithi132 ceph-mon[32020]: pgmap v333: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:20:01.065 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:00 smithi132 ceph-mon[32020]: overall HEALTH_OK 2024-03-18T23:20:01.066 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-18T23:20:01.076 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:20:01.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 -- 172.21.15.132:0/1157903722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230069f10 msgr2=0x7f023006a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 --2- 172.21.15.132:0/1157903722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230069f10 0x7f023006a370 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0224009900 tx=0x7f022402bcd0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 -- 172.21.15.132:0/1157903722 shutdown_connections 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 --2- 172.21.15.132:0/1157903722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f023006a8b0 0x7f023010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 --2- 172.21.15.132:0/1157903722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230069f10 0x7f023006a370 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 --2- 172.21.15.132:0/1157903722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0230107820 0x7f0230107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 -- 172.21.15.132:0/1157903722 >> 172.21.15.132:0/1157903722 conn(0x7f02300777c0 msgr2=0x7f0230077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 -- 172.21.15.132:0/1157903722 shutdown_connections 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 -- 172.21.15.132:0/1157903722 wait complete. 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.179+0000 7f0235d16640 1 Processor -- start 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 -- start start 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023006a8b0 0x7f0230100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111be0 con 0x7f0230107820 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111a60 con 0x7f023006a8b0 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f0235d16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111d60 con 0x7f0230069f10 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.132:46536/0 (socket says 172.21.15.132:46536) 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 -- 172.21.15.132:0/808710054 learned_addr learned my addr 172.21.15.132:0/808710054 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43054/0 (socket says 172.21.15.132:43054) 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022effd640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023006a8b0 0x7f0230100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 -- 172.21.15.132:0/808710054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023006a8b0 msgr2=0x7f0230100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023006a8b0 0x7f0230100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 -- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 msgr2=0x7f02301011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 -- 172.21.15.132:0/808710054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02301a6580 con 0x7f0230069f10 2024-03-18T23:20:01.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022ffff640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.180+0000 7f022f7fe640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9f0 tx=0x7f022000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.181+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022000cd80 con 0x7f0230069f10 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.181+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301a6870 con 0x7f0230069f10 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.181+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220010910 con 0x7f0230069f10 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.181+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200188f0 con 0x7f0230069f10 2024-03-18T23:20:01.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.181+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a6de0 con 0x7f0230069f10 2024-03-18T23:20:01.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.182+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0230102f40 con 0x7f0230069f10 2024-03-18T23:20:01.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.183+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f0220010ac0 con 0x7f0230069f10 2024-03-18T23:20:01.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.183+0000 7f022cff9640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 0x7f02180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.183+0000 7f022effd640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 0x7f02180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.183+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f02200cf5a0 con 0x7f0230069f10 2024-03-18T23:20:01.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.184+0000 7f022effd640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 0x7f02180a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f022402efd0 tx=0x7f022402cf90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.190 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.191+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f022009bcb0 con 0x7f0230069f10 2024-03-18T23:20:01.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:00 smithi196 ceph-mon[67089]: pgmap v333: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:20:01.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:00 smithi196 ceph-mon[67089]: overall HEALTH_OK 2024-03-18T23:20:01.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:00 smithi144 ceph-mon[66781]: pgmap v333: 97 pgs: 97 active+clean; 5.8 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-18T23:20:01.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:00 smithi144 ceph-mon[66781]: overall HEALTH_OK 2024-03-18T23:20:01.372 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.373+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0230109cc0 con 0x7f0230069f10 2024-03-18T23:20:01.375 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.376+0000 7f022cff9640 1 -- 172.21.15.132:0/808710054 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23173 (secure 0 0 0) 0x7f022009fd50 con 0x7f0230069f10 2024-03-18T23:20:01.375 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:01.375 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":99,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:19:08.892551+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4032784803":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6800/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3928210624":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3187950145":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2898017683":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 msgr2=0x7f02180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 0x7f02180a4e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f022402efd0 tx=0x7f022402cf90 comp rx=0 tx=0).stop 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 msgr2=0x7f0230100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f022000e9f0 tx=0x7f022000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 shutdown_connections 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f02180a2970 0x7f02180a4e30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0230107820 0x7f02301011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023006a8b0 0x7f0230100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 --2- 172.21.15.132:0/808710054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0230069f10 0x7f0230100750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 >> 172.21.15.132:0/808710054 conn(0x7f02300777c0 msgr2=0x7f02300ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 shutdown_connections 2024-03-18T23:20:01.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.378+0000 7f0235d16640 1 -- 172.21.15.132:0/808710054 wait complete. 2024-03-18T23:20:01.389 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:20:01.390 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 -- 172.21.15.132:0/2855274319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 msgr2=0x7f856806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 --2- 172.21.15.132:0/2855274319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f856806a350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8550009860 tx=0x7f855002bb70 comp rx=0 tx=0).stop 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 -- 172.21.15.132:0/2855274319 shutdown_connections 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 --2- 172.21.15.132:0/2855274319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f856806a890 0x7f856810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 --2- 172.21.15.132:0/2855274319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f856806a350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 --2- 172.21.15.132:0/2855274319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85681077c0 0x7f8568107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 -- 172.21.15.132:0/2855274319 >> 172.21.15.132:0/2855274319 conn(0x7f85680777f0 msgr2=0x7f8568077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 -- 172.21.15.132:0/2855274319 shutdown_connections 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.495+0000 7f856c827640 1 -- 172.21.15.132:0/2855274319 wait complete. 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 Processor -- start 2024-03-18T23:20:01.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 -- start start 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85681077c0 0x7f856819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111bd0 con 0x7f856806a890 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111a50 con 0x7f8568069ed0 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856c827640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111d50 con 0x7f85681077c0 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f855ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.132:50664/0 (socket says 172.21.15.132:50664) 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f856659c640 1 -- 172.21.15.132:0/2592326004 learned_addr learned my addr 172.21.15.132:0/2592326004 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f855ffff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:43072/0 (socket says 172.21.15.132:43072) 2024-03-18T23:20:01.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f8566d9d640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85681077c0 0x7f856819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f855ffff640 1 -- 172.21.15.132:0/2592326004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85681077c0 msgr2=0x7f856819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f855ffff640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85681077c0 0x7f856819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.496+0000 7f855ffff640 1 -- 172.21.15.132:0/2592326004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 msgr2=0x7f8568198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855ffff640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855ffff640 1 -- 172.21.15.132:0/2592326004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819d060 con 0x7f856806a890 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f856659c640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855ffff640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8550005940 tx=0x7f85500045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550041020 con 0x7f856806a890 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819d2f0 con 0x7f856806a890 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a4be0 con 0x7f856806a890 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855002c9f0 con 0x7f856806a890 2024-03-18T23:20:01.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.497+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550039040 con 0x7f856806a890 2024-03-18T23:20:01.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.498+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8548005290 con 0x7f856806a890 2024-03-18T23:20:01.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.499+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f8550044020 con 0x7f856806a890 2024-03-18T23:20:01.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.500+0000 7f855f7fe640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 0x7f85540a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.500+0000 7f856659c640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 0x7f85540a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.500+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f85500efd10 con 0x7f856806a890 2024-03-18T23:20:01.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.500+0000 7f856659c640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 0x7f85540a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85580096c0 tx=0x7f8558009260 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.504+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f855004b030 con 0x7f856806a890 2024-03-18T23:20:01.680 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.681+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8548005850 con 0x7f856806a890 2024-03-18T23:20:01.681 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.681+0000 7f855f7fe640 1 -- 172.21.15.132:0/2592326004 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f85500bcb80 con 0x7f856806a890 2024-03-18T23:20:01.681 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 msgr2=0x7f85540a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 0x7f85540a4e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85580096c0 tx=0x7f8558009260 comp rx=0 tx=0).stop 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 msgr2=0x7f85681985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8550005940 tx=0x7f85500045d0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 shutdown_connections 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f85540a2970 0x7f85540a4e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85681077c0 0x7f856819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856806a890 0x7f85681985b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.683 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 --2- 172.21.15.132:0/2592326004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8568069ed0 0x7f8568198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 >> 172.21.15.132:0/2592326004 conn(0x7f85680777f0 msgr2=0x7f85680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.683+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 shutdown_connections 2024-03-18T23:20:01.684 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.684+0000 7f856c827640 1 -- 172.21.15.132:0/2592326004 wait complete. 2024-03-18T23:20:01.694 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 -- 172.21.15.132:0/2893545348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c10be60 msgr2=0x7f617c10e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 --2- 172.21.15.132:0/2893545348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c10be60 0x7f617c10e2d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02bcd0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 -- 172.21.15.132:0/2893545348 shutdown_connections 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 --2- 172.21.15.132:0/2893545348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c10be60 0x7f617c10e2d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 --2- 172.21.15.132:0/2893545348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c10b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 --2- 172.21.15.132:0/2893545348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c102a90 0x7f617c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 -- 172.21.15.132:0/2893545348 >> 172.21.15.132:0/2893545348 conn(0x7f617c077880 msgr2=0x7f617c077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 -- 172.21.15.132:0/2893545348 shutdown_connections 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.799+0000 7f6184177640 1 -- 172.21.15.132:0/2893545348 wait complete. 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 Processor -- start 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 -- start start 2024-03-18T23:20:01.801 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c10be60 0x7f617c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111c30 con 0x7f617c102a90 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111ab0 con 0x7f617c10be60 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6184177640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f617c111db0 con 0x7f617c1033b0 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45772/0 (socket says 172.21.15.132:45772) 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 -- 172.21.15.132:0/2016612250 learned_addr learned my addr 172.21.15.132:0/2016612250 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181712640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6182714640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c10be60 0x7f617c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 -- 172.21.15.132:0/2016612250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 msgr2=0x7f617c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 -- 172.21.15.132:0/2016612250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c10be60 msgr2=0x7f617c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c10be60 0x7f617c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 -- 172.21.15.132:0/2016612250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c19d090 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181712640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.800+0000 7f6181f13640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f617400e9f0 tx=0x7f617400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.801+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617400cd80 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.801+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6174010910 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.801+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c19d380 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.801+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174018990 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.801+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1a4c10 con 0x7f617c102a90 2024-03-18T23:20:01.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.802+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c068a60 con 0x7f617c102a90 2024-03-18T23:20:01.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.803+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f6174010ac0 con 0x7f617c102a90 2024-03-18T23:20:01.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.803+0000 7f617affd640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 0x7f61640a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:01.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.803+0000 7f6181712640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 0x7f61640a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:01.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.803+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f61740cf5c0 con 0x7f617c102a90 2024-03-18T23:20:01.804 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.804+0000 7f6181712640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 0x7f61640a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f617c10db80 tx=0x7f6170009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:01.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.807+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f617409bcd0 con 0x7f617c102a90 2024-03-18T23:20:01.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.983+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f617c106f50 con 0x7f617c102a90 2024-03-18T23:20:01.983 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.984+0000 7f617affd640 1 -- 172.21.15.132:0/2016612250 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f617409fd70 con 0x7f617c102a90 2024-03-18T23:20:01.983 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 msgr2=0x7f61640a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 0x7f61640a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f617c10db80 tx=0x7f6170009500 comp rx=0 tx=0).stop 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 msgr2=0x7f617c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f617400e9f0 tx=0x7f617400eeb0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 shutdown_connections 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f61640a2970 0x7f61640a4e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.985 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f617c10be60 0x7f617c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f617c1033b0 0x7f617c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 --2- 172.21.15.132:0/2016612250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f617c102a90 0x7f617c1980f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:01.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 >> 172.21.15.132:0/2016612250 conn(0x7f617c077880 msgr2=0x7f617c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:01.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 shutdown_connections 2024-03-18T23:20:01.986 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:01.986+0000 7f6184177640 1 -- 172.21.15.132:0/2016612250 wait complete. 2024-03-18T23:20:01.995 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:01 smithi132 ceph-mon[32020]: from='client.16248 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:01.995 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:01 smithi132 ceph-mon[32020]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:20:01.995 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:01 smithi132 ceph-mon[32020]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:01.996 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/808710054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:01.996 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:01 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2592326004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:01.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:20:02.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 -- 172.21.15.132:0/4125936450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 msgr2=0x7f76a8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 --2- 172.21.15.132:0/4125936450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8101c30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7698009930 tx=0x7f769802bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 -- 172.21.15.132:0/4125936450 shutdown_connections 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 --2- 172.21.15.132:0/4125936450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 0x7f76a810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 --2- 172.21.15.132:0/4125936450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8101c30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 --2- 172.21.15.132:0/4125936450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 0x7f76a8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 -- 172.21.15.132:0/4125936450 >> 172.21.15.132:0/4125936450 conn(0x7f76a8078c20 msgr2=0x7f76a8079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 -- 172.21.15.132:0/4125936450 shutdown_connections 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 -- 172.21.15.132:0/4125936450 wait complete. 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.105+0000 7f76ad962640 1 Processor -- start 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- start start 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 0x7f76a8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 0x7f76a819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111bd0 con 0x7f76a81017b0 2024-03-18T23:20:02.107 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111a50 con 0x7f76a8102170 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8111d50 con 0x7f76a81077c0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45794/0 (socket says 172.21.15.132:45794) 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 -- 172.21.15.132:0/3368964720 learned_addr learned my addr 172.21.15.132:0/3368964720 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 0x7f76a8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a77fe640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 0x7f76a819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 -- 172.21.15.132:0/3368964720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 msgr2=0x7f76a819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 0x7f76a819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 -- 172.21.15.132:0/3368964720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 msgr2=0x7f76a8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 0x7f76a8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 -- 172.21.15.132:0/3368964720 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a819d100 con 0x7f76a81017b0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76a6ffd640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f769000e9f0 tx=0x7f769000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:02.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769000cd80 con 0x7f76a81017b0 2024-03-18T23:20:02.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7690005500 con 0x7f76a81017b0 2024-03-18T23:20:02.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a819d3f0 con 0x7f76a81017b0 2024-03-18T23:20:02.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690010690 con 0x7f76a81017b0 2024-03-18T23:20:02.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.106+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a4c80 con 0x7f76a81017b0 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.108+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8068a60 con 0x7f76a81017b0 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.109+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f76900046c0 con 0x7f76a81017b0 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.109+0000 7f76ac960640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 0x7f76940a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.109+0000 7f76a67fc640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 0x7f76940a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.109+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f76900cf1f0 con 0x7f76a81017b0 2024-03-18T23:20:02.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.110+0000 7f76a67fc640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 0x7f76940a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7698005ab0 tx=0x7f7698005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:02.112 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.113+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f769009bf50 con 0x7f76a81017b0 2024-03-18T23:20:02.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:01 smithi196 ceph-mon[67089]: from='client.16248 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:02.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:01 smithi196 ceph-mon[67089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:20:02.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:01 smithi196 ceph-mon[67089]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:02.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/808710054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:02.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:01 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2592326004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:02.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.289+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f76a802d1f0 con 0x7f76a81017b0 2024-03-18T23:20:02.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.290+0000 7f76ac960640 1 -- 172.21.15.132:0/3368964720 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f769009b850 con 0x7f76a81017b0 2024-03-18T23:20:02.289 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:20:02.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 msgr2=0x7f76940a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 0x7f76940a4e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7698005ab0 tx=0x7f7698005fd0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 msgr2=0x7f76a8198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f769000e9f0 tx=0x7f769000edf0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 shutdown_connections 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f76940a2970 0x7f76940a4e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76a81077c0 0x7f76a819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8102170 0x7f76a8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 --2- 172.21.15.132:0/3368964720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a81017b0 0x7f76a8198110 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 >> 172.21.15.132:0/3368964720 conn(0x7f76a8078c20 msgr2=0x7f76a8104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 shutdown_connections 2024-03-18T23:20:02.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.292+0000 7f76ad962640 1 -- 172.21.15.132:0/3368964720 wait complete. 2024-03-18T23:20:02.302 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-18T23:20:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:01 smithi144 ceph-mon[66781]: from='client.16248 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:01 smithi144 ceph-mon[66781]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-18T23:20:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:01 smithi144 ceph-mon[66781]: from='client.16254 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T23:20:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/808710054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:02.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:01 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2592326004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:02.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.405+0000 7fd32febf640 1 -- 172.21.15.132:0/1898190668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd328073bf0 msgr2=0x7fd328073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.405+0000 7fd32febf640 1 --2- 172.21.15.132:0/1898190668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd328073bf0 0x7fd328073fd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd324009ca0 tx=0x7fd32402c180 comp rx=0 tx=0).stop 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- 172.21.15.132:0/1898190668 shutdown_connections 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- 172.21.15.132:0/1898190668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd32810e080 0x7fd328110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- 172.21.15.132:0/1898190668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328074510 0x7fd32810db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- 172.21.15.132:0/1898190668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd328073bf0 0x7fd328073fd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- 172.21.15.132:0/1898190668 >> 172.21.15.132:0/1898190668 conn(0x7fd3280fd290 msgr2=0x7fd3280ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- 172.21.15.132:0/1898190668 shutdown_connections 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- 172.21.15.132:0/1898190668 wait complete. 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 Processor -- start 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- start start 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328073bf0 0x7fd32819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 0x7fd32819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113e00 con 0x7fd32810e080 2024-03-18T23:20:02.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113c80 con 0x7fd328074510 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.406+0000 7fd32febf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113f80 con 0x7fd328073bf0 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32dc5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328073bf0 0x7fd32819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32d45a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 0x7fd32819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45812/0 (socket says 172.21.15.132:45812) 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 -- 172.21.15.132:0/3802194337 learned_addr learned my addr 172.21.15.132:0/3802194337 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 -- 172.21.15.132:0/3802194337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328073bf0 msgr2=0x7fd32819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328073bf0 0x7fd32819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 -- 172.21.15.132:0/3802194337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 msgr2=0x7fd32819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 0x7fd32819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 -- 172.21.15.132:0/3802194337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32819f370 con 0x7fd32810e080 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32d45a640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 0x7fd32819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32e45c640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e940 tx=0x7fd32000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:02.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32000ccc0 con 0x7fd32810e080 2024-03-18T23:20:02.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd320005500 con 0x7fd32810e080 2024-03-18T23:20:02.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32819f660 con 0x7fd32810e080 2024-03-18T23:20:02.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320010690 con 0x7fd32810e080 2024-03-18T23:20:02.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.407+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281a6ef0 con 0x7fd32810e080 2024-03-18T23:20:02.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.408+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd328068610 con 0x7fd32810e080 2024-03-18T23:20:02.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.409+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fd3200046c0 con 0x7fd32810e080 2024-03-18T23:20:02.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.410+0000 7fd31effd640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 0x7fd3140a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:02.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.410+0000 7fd32dc5b640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 0x7fd3140a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:02.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.410+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fd3200cf110 con 0x7fd32810e080 2024-03-18T23:20:02.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.410+0000 7fd32dc5b640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 0x7fd3140a4e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd324004af0 tx=0x7fd324005c50 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:02.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.414+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd32009be70 con 0x7fd32810e080 2024-03-18T23:20:02.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.636+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd328108c70 con 0x7fd32810e080 2024-03-18T23:20:02.643 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.644+0000 7fd31effd640 1 -- 172.21.15.132:0/3802194337 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740371 (secure 0 0 0) 0x7fd32009b770 con 0x7fd32810e080 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "epoch": 25, 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "active_gid": 15294, 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "active_name": "x", 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "active_addrs": { 2024-03-18T23:20:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:6800", 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 4165175733 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:6801", 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 4165175733 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "active_addr": "172.21.15.132:6801/4165175733", 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "active_change": "2024-03-18T23:09:15.213102+0000", 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "active_mgr_features": 4540701547738038271, 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "available": true, 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "standbys": [ 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "gid": 15285, 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "z", 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "mgr_features": 4540701547738038271, 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:20:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-03-18T23:20:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-03-18T23:20:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-03-18T23:20:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:20:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:20:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-03-18T23:20:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-03-18T23:20:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-03-18T23:20:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-03-18T23:20:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:20:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.707 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.708 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:20:02.709 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:20:02.710 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:20:02.711 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:20:02.712 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:20:02.713 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:20:02.714 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:20:02.715 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:20:02.716 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:20:02.717 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:20:02.718 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.720 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.721 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.722 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.723 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.724 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.725 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.726 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.727 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.728 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.729 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.730 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.731 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:20:02.732 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.733 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.734 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.735 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.736 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.737 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:20:02.738 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:20:02.739 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.740 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.741 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-03-18T23:20:02.742 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.743 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:20:02.744 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.746 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.748 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:20:02.749 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.750 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:20:02.751 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:20:02.752 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.753 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.754 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:20:02.755 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.756 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.757 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.758 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.759 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.760 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.761 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.762 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.763 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.764 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.765 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.766 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.768 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.769 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:20:02.770 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.772 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.775 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.776 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:20:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.778 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:20:02.779 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-03-18T23:20:02.780 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:20:02.781 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:20:02.782 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.783 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.784 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:20:02.785 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.787 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.788 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.789 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.790 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.791 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.792 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.793 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.794 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.795 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.796 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.797 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.798 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.799 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:20:02.801 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.802 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.803 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.804 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.805 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.806 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:20:02.807 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:20:02.807 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.809 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.810 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.811 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-03-18T23:20:02.812 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.813 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-03-18T23:20:02.814 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:02.815 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.816 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.817 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.818 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.819 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.822 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.823 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.824 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.825 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.827 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.828 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.829 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.830 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-03-18T23:20:02.831 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.832 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.833 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.834 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.835 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.837 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.839 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.840 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.841 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.842 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.844 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.846 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.847 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.848 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.850 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.851 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.852 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.853 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.855 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.856 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.857 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.858 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.859 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.860 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.861 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.862 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.863 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-03-18T23:20:02.864 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:02.865 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.866 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.867 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.868 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.869 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.870 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.871 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.872 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.873 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.874 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.875 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.876 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.877 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.878 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.879 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.880 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.882 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.883 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.884 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.885 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.886 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.888 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.889 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.890 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.891 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.892 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.893 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.894 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.895 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-03-18T23:20:02.896 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-03-18T23:20:02.897 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.898 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.899 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.900 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.901 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.902 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.903 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.904 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.905 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.906 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.907 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.908 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.909 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.910 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.911 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.912 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.913 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.914 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.915 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.916 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.917 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.918 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.919 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.920 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.921 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.922 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-03-18T23:20:02.923 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.924 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.925 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.926 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.927 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.928 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.929 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.930 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.931 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.932 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.933 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.934 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.935 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.936 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.937 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.938 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.939 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.940 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.941 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.942 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-03-18T23:20:02.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.944 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.945 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.946 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.947 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.948 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.949 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.950 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-03-18T23:20:02.951 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:02.952 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.953 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.954 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.955 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.956 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.957 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.958 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.959 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:20:02.961 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.962 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:02.963 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.964 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.965 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.966 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.967 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.968 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.969 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.970 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.973 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.974 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.975 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.977 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.978 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.979 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.980 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.981 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.982 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.983 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.984 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.985 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.986 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.987 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.988 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.989 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.990 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.991 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.992 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.993 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.994 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:02.995 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.996 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.997 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.998 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:02.999 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.000 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.001 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.002 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.003 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.004 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-03-18T23:20:03.005 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.006 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.007 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.008 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.009 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.010 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.011 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.013 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-03-18T23:20:03.014 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.015 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.016 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.018 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.019 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.020 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.021 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.022 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.024 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.025 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.026 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.027 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.028 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.029 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.030 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.031 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.032 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.033 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.034 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.035 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-03-18T23:20:03.036 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.037 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.038 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.039 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.040 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.041 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.042 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.043 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:20:03.044 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:20:03.045 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.046 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.047 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:20:03.048 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.049 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.050 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.051 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.052 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.053 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.054 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.055 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.056 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.057 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.058 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.059 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.060 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-03-18T23:20:03.061 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-03-18T23:20:03.062 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.063 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-03-18T23:20:03.064 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.065 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.066 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.067 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.068 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.069 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.070 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.072 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.073 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.074 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.075 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:20:03.076 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.077 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.078 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.079 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.080 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.081 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.082 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.083 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.084 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.085 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.086 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.087 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.088 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.089 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.090 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.091 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.092 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "gid": 25148, 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "y", 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "mgr_features": 4540701547738038271, 2024-03-18T23:20:03.093 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.094 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.095 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.096 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.097 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.098 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.099 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.100 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-03-18T23:20:03.101 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-03-18T23:20:03.102 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.103 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.104 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.105 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.106 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.107 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.108 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.109 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.110 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.111 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.112 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.113 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-03-18T23:20:03.114 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-03-18T23:20:03.115 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.116 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.117 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.118 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.119 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.120 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.121 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:03.122 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-03-18T23:20:03.123 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:20:03.124 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.125 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:20:03.126 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.127 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.128 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-03-18T23:20:03.129 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.130 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.131 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.132 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.133 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.134 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-03-18T23:20:03.135 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.136 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.137 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-03-18T23:20:03.138 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.139 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.140 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.141 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.142 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.143 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.144 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.145 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.146 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.147 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-03-18T23:20:03.148 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.149 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.150 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.151 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.152 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.153 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.154 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.155 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.156 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.157 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.158 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-03-18T23:20:03.159 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.160 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.161 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.162 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.163 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.164 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.165 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.166 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.167 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.168 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.169 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.170 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.171 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.172 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.173 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.175 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.176 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.177 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.178 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.179 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.180 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.181 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.182 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.183 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.184 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.185 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.186 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.187 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.188 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.189 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.190 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-03-18T23:20:03.191 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.192 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.194 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.195 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.196 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.197 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.198 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.199 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.200 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.201 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.202 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.203 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.204 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.205 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.206 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.207 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.208 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.209 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.210 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.211 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.212 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.213 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.214 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.215 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.218 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.219 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.220 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.221 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.222 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.223 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.224 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.226 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.227 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.228 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.229 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.230 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.231 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.232 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.233 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.234 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.235 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.236 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.238 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.239 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.240 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.241 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.242 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.243 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.244 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.245 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.246 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.247 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.248 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.249 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.250 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.251 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.252 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.253 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.254 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.255 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.256 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.257 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.258 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.259 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.260 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.261 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.262 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.263 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-03-18T23:20:03.264 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.265 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.266 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.267 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.268 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.269 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.270 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.271 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-03-18T23:20:03.272 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.273 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.274 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.275 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.276 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-03-18T23:20:03.277 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.278 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.280 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.281 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.282 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.283 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.284 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.285 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-03-18T23:20:03.286 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.287 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.288 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.289 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.290 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.291 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.292 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.293 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.294 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.295 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.296 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.297 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.298 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.299 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.301 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.302 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.303 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.304 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.305 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.306 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.307 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.308 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.309 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.310 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.311 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.312 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-03-18T23:20:03.313 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-03-18T23:20:03.314 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.316 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.317 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.318 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.319 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.320 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.321 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.322 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.324 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.325 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.326 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.327 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.328 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.329 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.330 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.331 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.332 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.333 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.334 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.335 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.336 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.337 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.338 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.339 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.340 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.341 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.342 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-03-18T23:20:03.343 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.344 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.345 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.346 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.347 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.348 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.349 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.350 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.351 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.352 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.353 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.355 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.356 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.357 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.358 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.359 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.360 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.361 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.362 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.363 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.364 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.365 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.366 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.367 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.368 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.369 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.370 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.371 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.372 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.373 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.374 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.375 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.376 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.377 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.378 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.379 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.380 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.381 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.382 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.383 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.384 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:20:03.385 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.386 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.387 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.388 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.389 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.390 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.391 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.392 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.393 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.394 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.395 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.396 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.397 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.398 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.399 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.400 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.401 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.402 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.403 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.404 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.405 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.406 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.407 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.408 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.409 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.410 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.411 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.412 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.413 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.414 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.415 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.416 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.417 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.418 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.419 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.420 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.421 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.422 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.423 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.425 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.426 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.427 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.428 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.429 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.430 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.431 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.432 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.433 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.434 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.435 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.436 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.437 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.438 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.439 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.440 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.441 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.442 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.443 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.444 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-03-18T23:20:03.445 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.446 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-03-18T23:20:03.447 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-03-18T23:20:03.448 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.449 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:20:03.450 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.451 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.452 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.453 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.454 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.455 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.456 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.458 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.459 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.460 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.461 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.462 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.463 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.464 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.465 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.467 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-03-18T23:20:03.468 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.469 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.470 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.471 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.472 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.473 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.474 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.475 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.476 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.477 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:20:03.478 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:20:03.479 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:20:03.480 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.481 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.482 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.483 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.484 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.485 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.486 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.487 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.488 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.489 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.490 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.491 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.493 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.494 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.495 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.496 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.497 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.498 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.499 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.500 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.501 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.502 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.503 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.504 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.505 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.506 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.507 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.508 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-03-18T23:20:03.509 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.510 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.511 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.512 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-03-18T23:20:03.513 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.514 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.515 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.516 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.517 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.518 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.519 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.520 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.521 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-03-18T23:20:03.522 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:20:03.523 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "modules": [ 2024-03-18T23:20:03.524 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "dashboard", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "iostat", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "nfs", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "restful", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "snap_schedule" 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "available_modules": [ 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "alerts", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.525 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.526 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.527 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.528 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.529 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.530 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.531 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_destination": { 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_destination", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email address to send alerts to", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_from_name": { 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_from_name", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "Ceph", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.533 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Email From: name", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_host": { 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_host", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.534 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP server", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_password": { 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_password", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.535 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Password to authenticate with", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_port": { 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_port", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "465", 2024-03-18T23:20:03.536 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP port", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_sender": { 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_sender", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.537 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SMTP envelope sender", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_ssl": { 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_ssl", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.538 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "smtp_user": { 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "smtp_user", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.539 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "User to authenticate as", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "balancer", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.540 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "active": { 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "active", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.541 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_time": { 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_time", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0000", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.542 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "begin_weekday": { 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "begin_weekday", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T23:20:03.543 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_max_iterations": { 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_max_iterations", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "25", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "250", 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.544 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_metrics": { 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_metrics", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.545 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "crush_compat_step": { 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crush_compat_step", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.5", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0.001", 2024-03-18T23:20:03.546 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "0.999", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "aggressiveness of optimization", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "end_time": { 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_time", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2359", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.547 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "end_weekday": { 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "end_weekday", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.548 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "0", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "6", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.549 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.550 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.551 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.552 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.553 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "min_score": { 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_score", 2024-03-18T23:20:03.554 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:03.555 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "upmap", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "crush-compat", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "none", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "read", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap-read" 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Balancer mode", 2024-03-18T23:20:03.556 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_ids": { 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_ids", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T23:20:03.557 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T23:20:03.558 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_deviation": { 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_deviation", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.559 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "upmap_max_optimizations": { 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "upmap_max_optimizations", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.560 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm", 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.561 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_down_multiplier": { 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_down_multiplier", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.562 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_refresh_rate": { 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_refresh_rate", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "20", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.563 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "agent_starting_port": { 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "agent_starting_port", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4721", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_ptrace": { 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_ptrace", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T23:20:03.565 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_interval": { 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_interval", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T23:20:03.566 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "autotune_memory_target_ratio": { 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0.7", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.567 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm_log_destination": { 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cephadm_log_destination", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.568 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "file", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "file,syslog", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "syslog" 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "cgroups_split": { 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cgroups_split", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.569 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "config_checks_enabled": { 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_checks_enabled", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.570 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "config_dashboard": { 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_dashboard", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.571 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_alertmanager": { 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_alertmanager", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T23:20:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_base": { 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_base", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T23:20:03.573 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Container image name, without the tag", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_elasticsearch": { 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_elasticsearch", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.574 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "elasticsearch container image", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_grafana": { 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_grafana", 2024-03-18T23:20:03.575 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_haproxy": { 2024-03-18T23:20:03.576 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_haproxy", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "HAproxy container image", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_agent": { 2024-03-18T23:20:03.577 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_agent", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger agent container image", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.578 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_collector": { 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_collector", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger collector container image", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.579 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_jaeger_query": { 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_jaeger_query", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Jaeger query container image", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.580 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_keepalived": { 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_keepalived", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Keepalived container image", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.581 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_loki": { 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_loki", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Loki container image", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.582 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_node_exporter": { 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_node_exporter", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.583 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_nvmeof": { 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_nvmeof", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Nvme-of container image", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.584 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_prometheus": { 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_prometheus", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Prometheus container image", 2024-03-18T23:20:03.585 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_promtail": { 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_promtail", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Promtail container image", 2024-03-18T23:20:03.586 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "container_image_snmp_gateway": { 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_image_snmp_gateway", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.587 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "SNMP Gateway container image", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "container_init": { 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "container_init", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.588 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "daemon_cache_timeout": { 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "daemon_cache_timeout", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.589 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "default_cephadm_command_timeout": { 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.590 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "900", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "default_registry": { 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "default_registry", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.591 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "docker.io", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "device_cache_timeout": { 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_cache_timeout", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1800", 2024-03-18T23:20:03.592 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache device inventory", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "device_enhanced_scan": { 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_enhanced_scan", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.593 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "facts_cache_timeout": { 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "facts_cache_timeout", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.594 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "seconds to cache host facts data", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "host_check_interval": { 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "host_check_interval", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.595 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to perform a host check", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "hw_monitoring": { 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hw_monitoring", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.596 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "inventory_list_all": { 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "inventory_list_all", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.597 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.598 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "log_refresh_metadata": { 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_refresh_metadata", 2024-03-18T23:20:03.599 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.600 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.601 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.602 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.603 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.604 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "*", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.605 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "max_count_per_host": { 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_count_per_host", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.606 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of daemons per service per host", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "max_osd_draining_count": { 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_osd_draining_count", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T23:20:03.607 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "migration_current": { 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "migration_current", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "internal - do not modify", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.608 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "mode": { 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mode", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "root", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm-package", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "root" 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "oob_default_addr": { 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "oob_default_addr", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "169.254.1.1", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.610 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "prometheus_alerts_path": { 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus_alerts_path", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.611 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_insecure": { 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_insecure", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.612 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_password": { 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_password", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.613 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_url": { 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_url", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.614 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "registry_username": { 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "registry_username", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.615 INFO:teuthology.orchestra.run.smithi132.stdout: "secure_monitoring_stack": { 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "secure_monitoring_stack", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "service_discovery_port": { 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "service_discovery_port", 2024-03-18T23:20:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8765", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "cephadm service discovery port", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "ssh_config_file": { 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssh_config_file", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.617 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "use_agent": { 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_agent", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.618 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "use_repo_digest": { 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "use_repo_digest", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.619 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_failed_host_check": { 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_failed_host_check", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.620 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_daemons": { 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_daemons", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.621 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_on_stray_hosts": { 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_on_stray_hosts", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.622 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.623 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crash", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.624 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.625 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.626 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.627 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.629 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.630 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "retain_interval": { 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retain_interval", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "31536000", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T23:20:03.631 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_recent_interval": { 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_recent_interval", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1209600", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T23:20:03.632 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dashboard", 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T23:20:03.633 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T23:20:03.634 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T23:20:03.635 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_ENABLED": { 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T23:20:03.637 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.638 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.639 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.640 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.641 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.642 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.643 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.644 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.645 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.646 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.648 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.649 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_URL": { 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_URL", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.650 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.651 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.652 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.653 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.654 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.655 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "{}", 2024-03-18T23:20:03.656 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.657 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.658 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.659 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.660 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.661 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.662 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.663 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.664 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.665 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.666 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.667 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.668 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.669 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "45", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.670 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.671 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "admin", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.672 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.673 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.674 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.675 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.676 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T23:20:03.677 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "cross_origin_url": { 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cross_origin_url", 2024-03-18T23:20:03.678 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "crt_file": { 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "crt_file", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.679 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "debug": { 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "debug", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.680 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Enable/disable debug options", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "jwt_token_ttl": { 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "jwt_token_ttl", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "28800", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.681 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.682 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.683 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.684 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.685 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.686 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.687 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.688 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "motd": { 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "motd", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "The message of the day", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.689 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect_resolve_ip_addr": { 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.690 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.691 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8080", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.692 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.693 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl_server_port": { 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl_server_port", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8443", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.694 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "redirect", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.695 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "redirect" 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.696 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "url_prefix": { 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url_prefix", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.697 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.698 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_monitoring": { 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_monitoring", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "monitor device health metrics", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.699 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.700 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.701 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.702 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.703 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.704 INFO:teuthology.orchestra.run.smithi132.stdout: "mark_out_threshold": { 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mark_out_threshold", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "2419200", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.705 INFO:teuthology.orchestra.run.smithi132.stdout: "pool_name": { 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pool_name", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "device_health_metrics", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "retention_period": { 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "retention_period", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15552000", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long to retain device health metrics", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.707 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_frequency": { 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_frequency", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.708 INFO:teuthology.orchestra.run.smithi132.stdout: "self_heal": { 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "self_heal", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.709 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.710 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "warn_threshold": { 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "warn_threshold", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7257600", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.711 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "diskprediction_local", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.712 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.713 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.714 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.715 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.716 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "predict_interval": { 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predict_interval", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "86400", 2024-03-18T23:20:03.717 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "predictor_model": { 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "predictor_model", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.718 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "prophetstor", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.719 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "600", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "influx", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": false, 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "influxdb python module not found", 2024-03-18T23:20:03.720 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "batch_size": { 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "batch_size", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5000", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.721 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "database": { 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "database", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "ceph", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.722 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": { 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "hostname", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server hostname", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.723 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "30", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "5", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.724 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.725 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.726 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.727 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.728 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.729 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "password": { 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "password", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.730 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "password of InfluxDB server user", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "port": { 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "port", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "8086", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.731 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "InfluxDB server port", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "ssl": { 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ssl", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "false", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.732 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "threads": { 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threads", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "32", 2024-03-18T23:20:03.733 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "username": { 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "username", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.734 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "username of InfluxDB server user", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "verify_ssl": { 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "verify_ssl", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "true", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.735 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "insights", 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.736 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.737 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.738 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.739 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.740 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.741 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "iostat", 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.742 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.743 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.744 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.745 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.746 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.747 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "k8sevents", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_event_retention_days": { 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "ceph_event_retention_days", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "7", 2024-03-18T23:20:03.748 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "config_check_secs": { 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "config_check_secs", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.749 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "10", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.750 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.751 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.752 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.754 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.755 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "localpool", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "failure_domain": { 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "failure_domain", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "host", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.756 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "failure domain for any created local pool", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.757 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.758 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.759 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.760 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.761 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "min_size": { 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "min_size", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.762 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default min_size for any created local pool", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "num_rep": { 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "num_rep", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3", 2024-03-18T23:20:03.763 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default replica count for any created local pool", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_num": { 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_num", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "128", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.764 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "prefix": { 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prefix", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.765 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "name prefix for any created local pool", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "subtree": { 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "subtree", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "rack", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.766 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mds_autoscaler", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.767 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.768 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.770 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.771 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirroring", 2024-03-18T23:20:03.772 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.773 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.774 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.775 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.776 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.777 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "nfs", 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.778 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.781 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.782 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.783 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "fail_fs": { 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "fail_fs", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.784 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.785 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.786 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.787 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.788 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.789 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator": { 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "orchestrator", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "cephadm", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "rook", 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "test_orchestrator" 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.790 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Orchestrator backend", 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_perf_query", 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.791 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.792 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.794 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.795 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "osd_support", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.796 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.797 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.798 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.799 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.800 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.801 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "pg_autoscaler", 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.802 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.803 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.804 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.805 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.806 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.807 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.808 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "threshold": { 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "threshold", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "3.0", 2024-03-18T23:20:03.809 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1.0", 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "scaling threshold", 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "progress", 2024-03-18T23:20:03.810 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_pg_recovery_event": { 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_pg_recovery_event", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T23:20:03.811 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.812 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.813 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.814 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.815 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.816 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.817 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "max_completed_events": { 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_completed_events", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "50", 2024-03-18T23:20:03.818 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "number of past completed events to remember", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "sleep_interval": { 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "sleep_interval", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "5", 2024-03-18T23:20:03.819 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "how long the module is going to sleep", 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "prometheus", 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.820 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "cache": { 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "cache", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.821 INFO:teuthology.orchestra.run.smithi132.stdout: "exclude_perf_counters": { 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "exclude_perf_counters", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.822 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.823 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.824 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.825 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.826 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.827 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools": { 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.828 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.829 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "300", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "scrape_interval": { 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "scrape_interval", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.830 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15.0", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.831 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "::", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.832 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "9283", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.833 INFO:teuthology.orchestra.run.smithi132.stdout: "stale_cache_strategy": { 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stale_cache_strategy", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "log", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.834 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_behaviour": { 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_behaviour", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "default", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "default", 2024-03-18T23:20:03.835 INFO:teuthology.orchestra.run.smithi132.stdout: "error" 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "standby_error_status_code": { 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "standby_error_status_code", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "500", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "400", 2024-03-18T23:20:03.836 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "599", 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.837 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.838 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.839 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.840 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.841 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.842 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_snap_create": { 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_snap_create", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10", 2024-03-18T23:20:03.843 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "mirror_snapshot_schedule": { 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.844 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "trash_purge_schedule": { 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "trash_purge_schedule", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.845 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "restful", 2024-03-18T23:20:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "enable_auth": { 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enable_auth", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.847 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "key_file": { 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "key_file", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.848 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.849 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.850 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.851 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.852 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.853 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "server_addr": { 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_addr", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.854 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "server_port": { 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "server_port", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.855 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rgw", 2024-03-18T23:20:03.856 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.857 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.858 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.859 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.860 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.862 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.863 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rook", 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.864 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.865 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.866 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.867 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.868 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.869 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "storage_class": { 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "storage_class", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "local", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.870 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "selftest", 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.871 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.872 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.873 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.874 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.875 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.876 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "roption1": { 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption1", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.877 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "roption2": { 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "roption2", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.878 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "xyz", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption1": { 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption1", 2024-03-18T23:20:03.879 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption2": { 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption2", 2024-03-18T23:20:03.880 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption3": { 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption3", 2024-03-18T23:20:03.881 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "float", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption4": { 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption4", 2024-03-18T23:20:03.882 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption5": { 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption5", 2024-03-18T23:20:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption6": { 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption6", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.884 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "rwoption7": { 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rwoption7", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "1", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "42", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "testkey": { 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testkey", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.886 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "testlkey": { 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testlkey", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.887 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "testnewline": { 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "testnewline", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.888 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.889 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "allow_m_granularity": { 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "allow_m_granularity", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.890 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "dump_on_update": { 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "dump_on_update", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "dump database to debug log on update", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.891 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.892 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.893 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.894 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.895 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.896 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "stats", 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.897 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.898 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.899 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.900 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.901 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.902 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "status", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.904 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.905 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.906 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.907 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telegraf", 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "address": { 2024-03-18T23:20:03.908 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "address", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.909 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "15", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.910 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.911 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.912 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.913 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.914 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.915 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "telemetry", 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_basic": { 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_basic", 2024-03-18T23:20:03.916 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_crash": { 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_crash", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.917 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_device": { 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_device", 2024-03-18T23:20:03.918 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_ident": { 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_ident", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.919 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "channel_perf": { 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "channel_perf", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.920 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "contact": { 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "contact", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.921 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "description": { 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "description", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.922 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "device_url": { 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "device_url", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.923 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "enabled": { 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "enabled", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.924 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "24", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "8", 2024-03-18T23:20:03.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "last_opt_revision": { 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "last_opt_revision", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "1", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.926 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard": { 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.927 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "leaderboard_description": { 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "leaderboard_description", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.928 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.929 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.930 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.931 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.932 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.933 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "organization": { 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "organization", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.934 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "proxy": { 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "proxy", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.935 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "url": { 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "url", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T23:20:03.936 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "test_orchestrator", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.937 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.938 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.939 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.940 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.941 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.942 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.943 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.944 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.945 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.946 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.947 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "max_concurrent_clones": { 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "max_concurrent_clones", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "4", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.948 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "periodic_async_work": { 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "periodic_async_work", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Periodically check for async work", 2024-03-18T23:20:03.949 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_delay": { 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_delay", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "0", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T23:20:03.950 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "snapshot_clone_no_wait": { 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "True", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T23:20:03.951 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "can_run": true, 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "error_string": "", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "module_options": { 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "discovery_interval": { 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "discovery_interval", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "uint", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.952 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "100", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "identifier": { 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "identifier", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.953 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "interval": { 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "interval", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "secs", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.954 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "60", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "log_level": { 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_level", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.955 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.956 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster": { 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.957 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_cluster_level": { 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_cluster_level", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "info", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [ 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "critical", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "debug", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "error", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "info", 2024-03-18T23:20:03.958 INFO:teuthology.orchestra.run.smithi132.stdout: "warning" 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "log_to_file": { 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "log_to_file", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "bool", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 1, 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "False", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.959 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_host": { 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_host", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "", 2024-03-18T23:20:03.960 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_port": { 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_port", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "int", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "10051", 2024-03-18T23:20:03.961 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "zabbix_sender": { 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "zabbix_sender", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "str", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "level": "advanced", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "flags": 0, 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T23:20:03.962 INFO:teuthology.orchestra.run.smithi132.stdout: "min": "", 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "max": "", 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "enum_allowed": [], 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "desc": "", 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "long_desc": "", 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "tags": [], 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "see_also": [] 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "services": { 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "dashboard": "https://172.21.15.132:8443/" 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.963 INFO:teuthology.orchestra.run.smithi132.stdout: "always_on_modules": { 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "octopus": [ 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.964 INFO:teuthology.orchestra.run.smithi132.stdout: "pacific": [ 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "quincy": [ 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-03-18T23:20:03.965 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "reef": [ 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-03-18T23:20:03.966 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: ], 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "squid": [ 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "balancer", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "crash", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "devicehealth", 2024-03-18T23:20:03.967 INFO:teuthology.orchestra.run.smithi132.stdout: "orchestrator", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "pg_autoscaler", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "progress", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "rbd_support", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "status", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "telemetry", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "volumes" 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "last_failure_osd_epoch": 91, 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "active_clients": [ 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "devicehealth", 2024-03-18T23:20:03.968 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1380590778 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "libcephsqlite", 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.969 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3767351481 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "rbd_support", 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1507735958 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.970 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 798202367 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "snap_schedule", 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.971 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3159676973 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "name": "volumes", 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "addrvec": [ 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: { 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v2", 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "172.21.15.132:0", 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 314817233 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.972 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:03.973 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-03-18T23:20:03.973 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:02 smithi144 ceph-mon[66781]: pgmap v334: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:02 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/2016612250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:02 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3368964720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:02 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3802194337' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:02 smithi196 ceph-mon[67089]: pgmap v334: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-03-18T23:20:03.974 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:02 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/2016612250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:02 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3368964720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.975 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:02 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3802194337' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:20:03.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 msgr2=0x7fd3140a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:03.975 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 0x7fd3140a4e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd324004af0 tx=0x7fd324005c50 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 msgr2=0x7fd32819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd32000e940 tx=0x7fd32000ee00 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 shutdown_connections 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fd3140a2970 0x7fd3140a4e30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd32810e080 0x7fd32819ec40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd328074510 0x7fd32819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 --2- 172.21.15.132:0/3802194337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd328073bf0 0x7fd32819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 >> 172.21.15.132:0/3802194337 conn(0x7fd3280fd290 msgr2=0x7fd3280feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 shutdown_connections 2024-03-18T23:20:03.976 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:02.649+0000 7fd32febf640 1 -- 172.21.15.132:0/3802194337 wait complete. 2024-03-18T23:20:03.982 DEBUG:tasks.check_counter:Getting stats from a 2024-03-18T23:20:03.982 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-18T23:20:03.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:02 smithi132 ceph-mon[32020]: pgmap v334: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-03-18T23:20:03.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:02 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/2016612250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:02 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3368964720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:03.983 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:02 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3802194337' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T23:20:04.235 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:20:05.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:04 smithi132 ceph-mon[32020]: pgmap v335: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-03-18T23:20:05.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:04 smithi196 ceph-mon[67089]: pgmap v335: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-03-18T23:20:05.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:04 smithi144 ceph-mon[66781]: pgmap v335: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-03-18T23:20:06.103 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:05 smithi132 ceph-mon[32020]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:20:06.167 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.002137895, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000040331, 2024-03-18T23:20:06.168 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000758967, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000892883, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000297627 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.169 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 220, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 210, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1437120, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 128704, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.041811940, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.014054987, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.031914086, 2024-03-18T23:20:06.170 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004371164, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 209, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.009855599, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000047155 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1437120, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 128704 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:06.171 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 160, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 183728, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.013929371, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.009757601, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001766817, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 157, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.014523575, 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000092506 2024-03-18T23:20:06.172 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 183728 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-03-18T23:20:06.173 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149017, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 31, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:06.176 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-03-18T23:20:06.177 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 222, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1406118, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 222, 2024-03-18T23:20:06.178 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 222, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1406118, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:06.179 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:20:06.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:05 smithi196 ceph-mon[67089]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:20:06.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:05 smithi144 ceph-mon[66781]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T23:20:06.852 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-18T23:20:06.853 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-18T23:20:06.853 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-18T23:20:06.853 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-18T23:20:06.853 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-18T23:20:06.853 DEBUG:tasks.check_counter:Getting stats from d 2024-03-18T23:20:06.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-18T23:20:07.111 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:20:07.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:06 smithi132 ceph-mon[32020]: pgmap v337: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-03-18T23:20:07.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:06 smithi196 ceph-mon[67089]: pgmap v337: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-03-18T23:20:07.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:06 smithi144 ceph-mon[66781]: pgmap v337: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 230, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 218, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1441744, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 132992, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.043816126, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.014683245, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.031561154, 2024-03-18T23:20:08.669 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004681145, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 218, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.012484597, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000057268 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1441744, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 132992 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.670 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 165, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 184608, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.013762338, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.009693953, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001776289, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 162, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.014929812, 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000092159 2024-03-18T23:20:08.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 184608 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 0, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 0, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 0, 2024-03-18T23:20:08.672 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.000354644, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000000000, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000292816, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:08.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:08.674 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:08.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:08.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 149057, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 32, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-03-18T23:20:08.677 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-03-18T23:20:08.678 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 228, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1406898, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 228, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 228, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1406898, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:08.679 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:20:09.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:08 smithi196 ceph-mon[67089]: pgmap v338: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-03-18T23:20:09.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:08 smithi196 ceph-mon[67089]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:20:09.229 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:08 smithi132 ceph-mon[32020]: pgmap v338: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-03-18T23:20:09.229 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:08 smithi132 ceph-mon[32020]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:20:09.257 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-18T23:20:09.257 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-18T23:20:09.257 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-18T23:20:09.257 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-18T23:20:09.257 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-18T23:20:09.257 DEBUG:tasks.check_counter:Getting stats from g 2024-03-18T23:20:09.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-18T23:20:09.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:08 smithi144 ceph-mon[66781]: pgmap v338: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-03-18T23:20:09.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:08 smithi144 ceph-mon[66781]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T23:20:09.499 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:20:11.219 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:20:11.219 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:11.219 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-03-18T23:20:11.219 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-03-18T23:20:11.219 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.001478053, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000039660, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000594939, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000817810, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000272603 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.220 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 222, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 212, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1437664, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 129984, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:11.221 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.042984984, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.014765196, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.031930393, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004426536, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 211, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.010374310, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000049167 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:11.222 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1437664, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 129984 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 161, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 183904, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.014713952, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.009575679, 2024-03-18T23:20:11.223 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.002685700, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 158, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.018901701, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000119631 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:11.224 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 183904 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:11.225 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:11.226 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:11.227 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 148977, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 30, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:11.228 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:11.229 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 224, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1406378, 2024-03-18T23:20:11.230 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 224, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 224, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1406378, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:11.231 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:20:11.231 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:10 smithi132 ceph-mon[32020]: pgmap v340: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-03-18T23:20:11.232 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:10 smithi196 ceph-mon[67089]: pgmap v340: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-03-18T23:20:11.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:10 smithi144 ceph-mon[66781]: pgmap v340: 97 pgs: 97 active+clean; 5.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-03-18T23:20:11.950 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-18T23:20:11.950 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-18T23:20:11.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-18T23:20:11.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-18T23:20:11.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-18T23:20:11.951 DEBUG:tasks.check_counter:Getting stats from j 2024-03-18T23:20:11.951 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-18T23:20:12.193 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T23:20:13.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:12 smithi196 ceph-mon[67089]: pgmap v341: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-18T23:20:13.286 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:12 smithi132 ceph-mon[32020]: pgmap v341: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-18T23:20:13.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:12 smithi144 ceph-mon[66781]: pgmap v341: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 4, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 4, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2992, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 448, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.001377740, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.000037553, 2024-03-18T23:20:13.849 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.000578318, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000728222, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000242740 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-18T23:20:13.850 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 228, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 218, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1439296, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 133824, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.042515128, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.014652786, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.031644706, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.004484754, 2024-03-18T23:20:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 217, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.011477316, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000052890 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 1439296, 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 133824 2024-03-18T23:20:13.852 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 166, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 184784, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.014724093, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.010476112, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.001751283, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:13.853 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 163, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.014552345, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000089278 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 184784 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 0, 2024-03-18T23:20:13.854 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:13.855 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:13.856 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 148977, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 30, 2024-03-18T23:20:13.857 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2024-03-18T23:20:13.858 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 2, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 230, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1407158, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 230, 2024-03-18T23:20:13.859 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 230, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1407158, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2024-03-18T23:20:13.860 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:13.861 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-03-18T23:20:13.861 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-03-18T23:20:14.514 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-18T23:20:14.514 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-18T23:20:14.514 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-18T23:20:14.514 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-18T23:20:14.514 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-18T23:20:14.514 DEBUG:tasks.check_counter:Getting stats from b 2024-03-18T23:20:14.515 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-18T23:20:14.771 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:20:15.161 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:14 smithi144 ceph-mon[66781]: pgmap v342: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-03-18T23:20:15.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:14 smithi196 ceph-mon[67089]: pgmap v342: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-03-18T23:20:15.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:14 smithi132 ceph-mon[32020]: pgmap v342: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-03-18T23:20:16.405 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-18T23:20:16.405 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:16.405 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1525, 2024-03-18T23:20:16.405 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1334, 2024-03-18T23:20:16.405 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 509223, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 4374303, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 100, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 7, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.122935550, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.033042097, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.054245861, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.028165327, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1335, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.343567580, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000257353 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:16.406 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1462, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000224588, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000153 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 1136 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1249, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 649, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 477358, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 943696, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 15, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-18T23:20:16.407 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.099776824, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.023074159, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.033391435, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.039965509, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 646, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.314791300, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000487293 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1107, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000154946, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000139 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.408 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 101152, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 488272 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1933, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1143, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 2165161, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 980765, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 17, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 8, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.151079192, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.042427116, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.075585721, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.043126995, 2024-03-18T23:20:16.409 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1140, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.333686703, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000292707 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1724, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000250881, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000145 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1827744, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 240864 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-18T23:20:16.410 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1206, 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.187003322, 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000155060 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000007910, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000003955 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 178, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 177, 2024-03-18T23:20:16.412 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 177, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.173813789, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000981998 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 1, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 297, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 38, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 4, 2024-03-18T23:20:16.413 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 22, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 12, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 32, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 6, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 3, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 3, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 3, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 225, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 209, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 1, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-18T23:20:16.414 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 367, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 2, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 5, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 1, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 11, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 4, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 234, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 6, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:16.415 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 3, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 21, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 20, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 7, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 29, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 12, 2024-03-18T23:20:16.416 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 1, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 28 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 6, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 2, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 2, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-18T23:20:16.417 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 2, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:16.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 267, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 246, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 246, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 21, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 2, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 50, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 49, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 49, 2024-03-18T23:20:16.419 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 1, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 1, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 1, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 6670869, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 6822052, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 232, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.377353991, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001626525 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.420 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 32, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 37, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 5, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 19, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 20, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 1, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 38, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 44, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 6, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 3, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 19, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 16, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 53516, 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 231804 2024-03-18T23:20:16.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 251, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 3, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 178, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 33, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 8, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.004201056, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000840211 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-18T23:20:16.422 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 112, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.017631607, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000157425 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000796029, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000265343 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-03-18T23:20:16.424 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000138468, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000138468 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.425 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 11, 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001442577, 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000131143 2024-03-18T23:20:16.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.011085297, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001385662 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 11, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.046251521, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004204683 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000238407, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000238407 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000647106, 2024-03-18T23:20:16.428 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000647106 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002887897, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001443948 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.429 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.019465687, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.009732843 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.019373363, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.003874672 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:16.430 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.033280764, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004160095 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:16.431 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 7, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.016374010, 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002339144 2024-03-18T23:20:16.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.433 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 2, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 5, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 3, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 31, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 15, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 640, 2024-03-18T23:20:16.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 120, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 120, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:16.435 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:16.436 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 351042, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 152, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 352, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 4, 2024-03-18T23:20:16.437 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 167, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-18T23:20:16.438 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 167242, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 1445, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 1315, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 3640203, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 32, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 1315, 2024-03-18T23:20:16.439 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1315, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 18.272697523, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.013895587 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1315, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1952 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 1315, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 1315, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-18T23:20:16.440 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 277, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 25, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 232, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 168, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 506, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:16.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 50, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 7, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-18T23:20:16.442 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-18T23:20:16.443 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 101, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 14, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 12, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 677, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 10 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-18T23:20:16.444 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:16.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 4703, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 2661647, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 4703, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 4703, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2661647, 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:16.446 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1315, 2024-03-18T23:20:16.447 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 3640203, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1315, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3640203, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:16.448 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1315, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1315, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1315, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1315, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:16.449 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5637ad988500": { 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 267, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 2627748, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 267, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 232, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2627748, 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:16.451 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:16.452 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:16.452 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:16.452 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.452 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:16.452 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-18T23:20:17.106 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-03-18T23:20:17.107 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-18T23:20:17.107 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-18T23:20:17.107 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-03-18T23:20:17.107 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-03-18T23:20:17.107 DEBUG:tasks.check_counter:Getting stats from e 2024-03-18T23:20:17.107 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-18T23:20:17.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:16 smithi196 ceph-mon[67089]: pgmap v343: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-03-18T23:20:17.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:16 smithi144 ceph-mon[66781]: pgmap v343: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-03-18T23:20:17.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:16 smithi132 ceph-mon[32020]: pgmap v343: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-03-18T23:20:17.373 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:20:19.028 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:18 smithi144 ceph-mon[66781]: pgmap v344: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-18T23:20:19.028 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-18T23:20:19.028 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:19.028 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 252, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 237, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1483282, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 137268, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.036042890, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008414195, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.034274418, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003551263, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 236, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.007323169, 2024-03-18T23:20:19.029 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000031030 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1482448, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 136944 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 697, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 859, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 173320, 2024-03-18T23:20:19.030 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 674015, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.052809926, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.017200532, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.012243507, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.018113307, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 857, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.029100118, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033955 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.031 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 454272 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1570, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1572, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 476565, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 499318, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.098675129, 2024-03-18T23:20:19.032 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.033470283, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.026031915, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.031297440, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1570, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.033188261, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000021139 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:19.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3792, 2024-03-18T23:20:19.034 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.076232642, 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020103 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:19.036 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 6, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 2, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 4, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 3, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 3, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-18T23:20:19.037 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-18T23:20:19.038 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:20:19.039 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-18T23:20:19.040 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-18T23:20:19.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 9, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-18T23:20:19.042 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4201870, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4201729, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.043 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 9 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 8, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 8, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 6, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 6, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 6, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 6, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-18T23:20:19.044 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 47456, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.045 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-18T23:20:19.046 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.047 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.048 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-18T23:20:19.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-18T23:20:19.050 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.051 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.052 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.053 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.054 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 3, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 2, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-03-18T23:20:19.056 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 6653, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:19.057 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:19.058 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:19.059 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 189410, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 52, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 167, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-18T23:20:19.060 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 31613, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 104, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 2262, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 2262, 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-18T23:20:19.061 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2262, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.053983143, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000465951 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2262, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2263 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 2262, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2262, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-18T23:20:19.062 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1469, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 791, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-18T23:20:19.063 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-18T23:20:19.064 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 101, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 13, 2024-03-18T23:20:19.065 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.066 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:19.067 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 2515, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1890983, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 2515, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2515, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1890983, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2262, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 7571, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2262, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 7571, 2024-03-18T23:20:19.069 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:19.070 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2262, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2262, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2262, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2262, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:19.071 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55c131c4a6e0": { 2024-03-18T23:20:19.072 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:19.073 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:19.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:19.074 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-18T23:20:19.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:18 smithi196 ceph-mon[67089]: pgmap v344: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-18T23:20:19.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:18 smithi132 ceph-mon[32020]: pgmap v344: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-03-18T23:20:19.815 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-18T23:20:19.815 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-18T23:20:19.815 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-18T23:20:19.815 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:20:19.815 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:20:19.815 DEBUG:tasks.check_counter:Getting stats from h 2024-03-18T23:20:19.815 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-18T23:20:20.075 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:20:21.222 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:20 smithi144 ceph-mon[66781]: pgmap v345: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-18T23:20:21.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:20 smithi196 ceph-mon[67089]: pgmap v345: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-18T23:20:21.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:20 smithi132 ceph-mon[32020]: pgmap v345: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 344, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 539, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 108725, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 248895, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 8, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.037509096, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.017322088, 2024-03-18T23:20:21.597 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.015358569, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.002934242, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 541, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.027572149, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000050965 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 817, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000149406, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000182 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:21.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 75, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 234, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 39052, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 522977, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 11, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.013342240, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.006057731, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.006028874, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000230772, 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:21.599 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 234, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.023242497, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000099326 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 90, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000013776, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000153 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 455712 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 299, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 306, 2024-03-18T23:20:21.600 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1493357, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 171729, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 11, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 7, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.042545156, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.011615845, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.039908586, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003882233, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 299, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.011609044, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000038826 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:21.601 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 52, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000008644, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000166 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1490192, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 141216 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:21.602 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 64, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001208539, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000018883 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000006364, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000003182 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-18T23:20:21.603 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 1, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 1, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-18T23:20:21.604 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 16, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 16, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 13, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 3, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 1, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 1, 2024-03-18T23:20:21.605 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 1, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 3, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 1, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 3, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 117, 2024-03-18T23:20:21.606 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:20:21.607 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 2, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-18T23:20:21.608 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-18T23:20:21.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 9, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 9, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-03-18T23:20:21.610 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4201870, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012803025, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001600378 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 18, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 18, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 16, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 17, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 1, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 16, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 16, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-18T23:20:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49692, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223612 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 1, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 1, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 17, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-18T23:20:21.613 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-18T23:20:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.619 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-18T23:20:21.620 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.621 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.623 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:21.624 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 3, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 2, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 660, 2024-03-18T23:20:21.625 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:21.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:21.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 230167, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 72, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:21.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 167, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 73981, 2024-03-18T23:20:21.629 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 306, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 46, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 18048, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 27, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 46, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 46, 2024-03-18T23:20:21.630 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.197996693, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004304275 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 46, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 68 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 46, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 46, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-03-18T23:20:21.631 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 8, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 9, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 6, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-03-18T23:20:21.632 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-18T23:20:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 101, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 13, 2024-03-18T23:20:21.634 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 8, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 8, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 14, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 1 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-18T23:20:21.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:21.636 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 714, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1559652, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 714, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 714, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1559652, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.637 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 46, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 18048, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 46, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 18048, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.638 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 46, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 46, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 46, 2024-03-18T23:20:21.639 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 46, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:21.640 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:21.641 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55fc084ba500": { 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 9, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 7566, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 9, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 8, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 7566, 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:21.642 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:21.643 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:21.643 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:21.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:21.643 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-18T23:20:22.450 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-18T23:20:22.450 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-03-18T23:20:22.450 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-18T23:20:22.451 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:20:22.451 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:20:22.451 DEBUG:tasks.check_counter:Getting stats from k 2024-03-18T23:20:22.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-18T23:20:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T23:20:23.102 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:22 smithi144 ceph-mon[66781]: pgmap v346: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-03-18T23:20:23.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:22 smithi196 ceph-mon[67089]: pgmap v346: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-03-18T23:20:23.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:22 smithi132 ceph-mon[32020]: pgmap v346: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-03-18T23:20:24.470 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 0, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 0, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 0, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.001048185, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.000000000, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.000247085, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:24.471 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:24.472 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 231, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 221, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1440112, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 135744, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.034341415, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008369081, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.032058640, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003764275, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 220, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.006665255, 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000030296 2024-03-18T23:20:24.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1440112, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 135744 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 6, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 171, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 3312, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 185760, 2024-03-18T23:20:24.474 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.008297189, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.004805700, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.001991840, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 169, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.013727508, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000081227 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:24.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 185760 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:24.476 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:24.477 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:24.478 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 148977, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 30, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 0, 2024-03-18T23:20:24.479 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 2, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-18T23:20:24.480 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 233, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1407548, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 233, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 233, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1407548, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-18T23:20:24.481 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-18T23:20:25.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:24 smithi144 ceph-mon[66781]: pgmap v347: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:25.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-18T23:20:25.122 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-18T23:20:25.123 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-18T23:20:25.123 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-18T23:20:25.123 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-18T23:20:25.123 DEBUG:tasks.check_counter:Getting stats from c 2024-03-18T23:20:25.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-18T23:20:25.222 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:24 smithi196 ceph-mon[67089]: pgmap v347: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:25.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:24 smithi132 ceph-mon[32020]: pgmap v347: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:25.373 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:20:27.051 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:26 smithi196 ceph-mon[67089]: pgmap v348: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1724, 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1725, 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3136568, 2024-03-18T23:20:27.351 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 547924, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.102694989, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.028885754, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.030125810, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.034083466, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1723, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.036973236, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000021458 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.352 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 261, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 244, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1487528, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 139530, 2024-03-18T23:20:27.353 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.034048168, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008352786, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.032505806, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.003678304, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 244, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.011407016, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000046750 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.354 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1485872, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 138192 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 770, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 935, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 189319, 2024-03-18T23:20:27.355 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 701457, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.058686956, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.021231438, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.012783323, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.018925810, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 932, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.027537356, 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000029546 2024-03-18T23:20:27.356 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 457232 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-03-18T23:20:27.357 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4174, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.080384123, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000019258 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.358 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-03-18T23:20:27.359 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 27, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 3, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 24, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 23, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-03-18T23:20:27.360 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-03-18T23:20:27.361 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:20:27.362 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-03-18T23:20:27.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:27.364 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 267, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-03-18T23:20:27.365 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 50, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 6670869, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 6822052, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 6801151, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.366 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 267 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 24, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 28, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 4, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 10, 2024-03-18T23:20:27.367 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 11, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 27, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 32, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 5, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50008, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215420 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-03-18T23:20:27.368 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.369 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-03-18T23:20:27.370 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.371 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.372 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-03-18T23:20:27.373 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.375 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:26 smithi132 ceph-mon[32020]: pgmap v348: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:27.375 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:26 smithi144 ceph-mon[66781]: pgmap v348: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.375 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.376 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.377 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.378 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.379 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.380 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-03-18T23:20:27.381 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-03-18T23:20:27.382 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 2, 2024-03-18T23:20:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 5, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 4, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 6675, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:27.384 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:27.385 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:27.386 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 190318, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 76, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 167, 2024-03-18T23:20:27.387 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 125987, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 1017, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-03-18T23:20:27.388 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2495, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 26, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 4, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2495, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2495, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.919981020, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000368729 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2495, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2496 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2495, 2024-03-18T23:20:27.389 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2495, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 1555, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 937, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-03-18T23:20:27.390 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:27.391 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-03-18T23:20:27.392 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 101, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 13, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 5, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-03-18T23:20:27.393 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-03-18T23:20:27.394 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2751, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 4549893, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2751, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2751, 2024-03-18T23:20:27.395 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 4549893, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:27.396 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2495, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2627758, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2495, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2627758, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:27.397 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2495, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2495, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2495, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2495, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.398 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.399 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55ad80a506e0": { 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:27.400 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:27.401 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-18T23:20:28.104 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-18T23:20:28.105 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-18T23:20:28.105 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-18T23:20:28.105 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:20:28.105 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:20:28.105 DEBUG:tasks.check_counter:Getting stats from f 2024-03-18T23:20:28.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-18T23:20:28.351 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:20:29.225 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:28 smithi196 ceph-mon[67089]: pgmap v349: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-18T23:20:29.357 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:28 smithi144 ceph-mon[66781]: pgmap v349: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-18T23:20:29.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:28 smithi132 ceph-mon[32020]: pgmap v349: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-18T23:20:29.913 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-18T23:20:29.913 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2884, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2886, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3045307, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 917119, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.152552380, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.046976783, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039951886, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.054195978, 2024-03-18T23:20:29.914 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2884, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.054629995, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000018942 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:29.915 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 8, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 172, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 3868, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 458554, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.008284270, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.004554152, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.002296587, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000061431, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 170, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.011057593, 2024-03-18T23:20:29.916 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000065044 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 457856 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 254, 2024-03-18T23:20:29.917 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 239, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1483260, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 138542, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.034002913, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008453649, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.032255056, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.003708975, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 238, 2024-03-18T23:20:29.918 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.009097412, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000038224 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1482992, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 138224 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-03-18T23:20:29.919 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5068, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.080028777, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000015790 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.920 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:29.921 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 2, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 2, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 2, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-03-18T23:20:29.922 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:29.923 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T23:20:29.924 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-03-18T23:20:29.925 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:29.926 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1928, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-03-18T23:20:29.927 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 6445784, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 6441602, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.928 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 1928 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 5, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 23, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 18, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 2, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 11, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 9, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 2, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 21, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 19, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-03-18T23:20:29.929 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 52524, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215420 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.930 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.931 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-03-18T23:20:29.932 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:29.933 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.934 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.935 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-03-18T23:20:29.935 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.935 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.936 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.936 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.936 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.937 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.938 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:29.939 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.940 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.941 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.943 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 6677, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:29.944 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:29.945 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:29.946 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 181350, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 48, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:29.947 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 167, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 18150, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 140, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.948 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2887, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2887, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2887, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.099977371, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000381010 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2887, 2024-03-18T23:20:29.949 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2888 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2887, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 2887, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 1489, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1396, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-03-18T23:20:29.950 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:29.951 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-03-18T23:20:29.952 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 101, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 13, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 4, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 4, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-03-18T23:20:29.953 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-03-18T23:20:29.954 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3142, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 4233717, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:29.955 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3142, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3142, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 4233717, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:29.956 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2887, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2251485, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2887, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2251485, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.957 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2887, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2887, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2887, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2887, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.958 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-03-18T23:20:29.959 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55b0316d86e0": { 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:29.960 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:29.961 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-18T23:20:30.726 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-18T23:20:30.726 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-18T23:20:30.726 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-18T23:20:30.726 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:20:30.726 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:20:30.726 DEBUG:tasks.check_counter:Getting stats from i 2024-03-18T23:20:30.726 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-18T23:20:31.017 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:20:31.257 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:30 smithi196 ceph-mon[67089]: pgmap v350: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-18T23:20:31.257 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:30 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:20:31.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:30 smithi144 ceph-mon[66781]: pgmap v350: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-18T23:20:31.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:30 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:20:31.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:30 smithi132 ceph-mon[32020]: pgmap v350: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-18T23:20:31.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:30 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T23:20:32.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:32 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:32 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:32 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:20:32.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:32 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:20:32.427 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:32 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:32 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:32 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:32 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:20:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:32 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:20:32.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:32 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:32 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:32 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:32 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T23:20:32.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:32 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T23:20:32.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:32 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' 2024-03-18T23:20:32.668 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-18T23:20:32.668 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:32.668 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 3105, 2024-03-18T23:20:32.668 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 3351, 2024-03-18T23:20:32.668 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 800103, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 3705084, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.190305150, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.073963717, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.055850235, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.049133021, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3349, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.081391359, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000024303 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:32.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1282, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000190124, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000148 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 331216 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1629, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4017, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1918403, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 1462260, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 10, 2024-03-18T23:20:32.670 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.150045333, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.064863350, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.097797355, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.003826957, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4017, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.066806947, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000016631 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3874, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000687724, 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000177 2024-03-18T23:20:32.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1498384, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 143808 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 86, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 117, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 40496, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 201388, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.009658781, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.002641122, 2024-03-18T23:20:32.672 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.006165047, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000655049, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 114, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.011445549, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000100399 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 121, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000017416, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000143 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 130496 2024-03-18T23:20:32.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4386, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.772579515, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000176146 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.674 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000263762, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000016485 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 603, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 600, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 600, 2024-03-18T23:20:32.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.917807915, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003196346 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 3, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 16, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 15, 2024-03-18T23:20:32.676 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 15, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 15, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 2, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 2, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 612, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 603, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 1, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 3, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-03-18T23:20:32.677 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 1, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 11, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 1, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 2, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 3, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 117, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T23:20:32.678 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 603, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 600, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 592, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 673, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 661, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 2, 2024-03-18T23:20:32.679 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 597 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 4, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 1, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 1, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-03-18T23:20:32.680 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 1, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-03-18T23:20:32.681 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 1929, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 1929, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 3, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 3, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-03-18T23:20:32.682 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 6446383, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1853, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2.787801572, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001504480 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 19, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 32, 2024-03-18T23:20:32.683 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 13, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 18, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 21, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 3, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 15, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 31, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 16, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 2, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 7, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 5, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 52660, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223612 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.684 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 1796, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 1, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 603, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 17, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 5, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000317509, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000317509 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001073855, 2024-03-18T23:20:32.685 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000357951 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000336504, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000336504 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000190758, 2024-03-18T23:20:32.687 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000190758 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.688 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.689 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.690 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001152065, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001152065 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-03-18T23:20:32.691 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.692 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.693 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 592, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.911955480, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.003229654 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002781744, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002781744 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.696 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 87, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 87, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 684, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:32.697 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:32.698 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:32.699 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 310172, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 114, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:32.700 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8032, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 167, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 84131, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 463, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-03-18T23:20:32.701 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 2542, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 2267808, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 2542, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2542, 2024-03-18T23:20:32.702 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 4.053916818, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001594774 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2542, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 2573 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 2542, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 1, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 2541, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-03-18T23:20:32.706 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 1854, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 50, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 13, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-03-18T23:20:32.707 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 592, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-03-18T23:20:32.708 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 101, 2024-03-18T23:20:32.709 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 13, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 29, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 4 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-03-18T23:20:32.710 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 1, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 1, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 1, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 1, 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-03-18T23:20:32.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 4816, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2307643, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 4816, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 4816, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2307643, 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:32.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2542, 2024-03-18T23:20:32.713 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2267949, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2542, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2267949, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-03-18T23:20:32.714 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 2542, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 2542, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2542, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2542, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-03-18T23:20:32.715 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 141, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 141, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55b102080500": { 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 1929, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2252079, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 1929, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 1853, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2252079, 2024-03-18T23:20:32.717 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:32.718 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-18T23:20:33.372 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-18T23:20:33.372 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-03-18T23:20:33.372 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-03-18T23:20:33.372 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-18T23:20:33.373 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T23:20:33.373 DEBUG:tasks.check_counter:Getting stats from l 2024-03-18T23:20:33.373 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-18T23:20:33.398 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:33 smithi196 ceph-mon[67089]: pgmap v351: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:33.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:33 smithi144 ceph-mon[66781]: pgmap v351: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:33.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:33 smithi132 ceph-mon[32020]: pgmap v351: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:33.617 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T23:20:35.266 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-18T23:20:35.266 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T23:20:35.266 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 4, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 4, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2992, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 448, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 0, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.001448014, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.000042326, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.000679540, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000804632, 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000268210 2024-03-18T23:20:35.267 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 171, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 320, 2024-03-18T23:20:35.268 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 185664, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.007626890, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.004653458, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.001471504, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 168, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014072741, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000083766 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:35.269 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 185664 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 235, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 225, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1441200, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 138304, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 2, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 1, 2024-03-18T23:20:35.270 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.034776755, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.008601593, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.032179947, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.003761064, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 224, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.007618049, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000034009 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1441200, 2024-03-18T23:20:35.271 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 138304 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T23:20:35.272 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T23:20:35.273 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T23:20:35.274 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 149017, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 31, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-03-18T23:20:35.275 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 928, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 2, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-03-18T23:20:35.276 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 237, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1408068, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 237, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 237, 2024-03-18T23:20:35.277 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1408068, 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-18T23:20:35.278 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-18T23:20:35.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:35 smithi144 ceph-mon[66781]: pgmap v352: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:35.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:35 smithi132 ceph-mon[32020]: pgmap v352: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:35.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:35 smithi196 ceph-mon[67089]: pgmap v352: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:36.047 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-18T23:20:36.047 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-18T23:20:36.047 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-18T23:20:36.047 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-18T23:20:36.048 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-18T23:20:36.048 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-18T23:20:36.059 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T23:20:36.059 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:20:36.059 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:20:36.080 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-03-18T23:20:36.080 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T23:20:36.139 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 -- 172.21.15.132:0/901940569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41077f0 msgr2=0x7f37f4107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 --2- 172.21.15.132:0/901940569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41077f0 0x7f37f4107bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009890 tx=0x7f37f002bb70 comp rx=0 tx=0).stop 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 -- 172.21.15.132:0/901940569 shutdown_connections 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 --2- 172.21.15.132:0/901940569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 --2- 172.21.15.132:0/901940569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41017e0 0x7f37f4101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 --2- 172.21.15.132:0/901940569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41077f0 0x7f37f4107bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 -- 172.21.15.132:0/901940569 >> 172.21.15.132:0/901940569 conn(0x7f37f40fd290 msgr2=0x7f37f40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 -- 172.21.15.132:0/901940569 shutdown_connections 2024-03-18T23:20:36.299 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.297+0000 7f37fb953640 1 -- 172.21.15.132:0/901940569 wait complete. 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 Processor -- start 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 -- start start 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 0x7f37f4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41077f0 0x7f37f419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4111c00 con 0x7f37f41021a0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4111a80 con 0x7f37f41077f0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37fb953640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4111d80 con 0x7f37f41017e0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37f8eee640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37f96ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 0x7f37f4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37f8eee640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45116/0 (socket says 172.21.15.132:45116) 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37f8eee640 1 -- 172.21.15.132:0/1945484395 learned_addr learned my addr 172.21.15.132:0/1945484395 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.298+0000 7f37f9ef0640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41077f0 0x7f37f419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 -- 172.21.15.132:0/1945484395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41077f0 msgr2=0x7f37f419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41077f0 0x7f37f419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 -- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 msgr2=0x7f37f4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 -- 172.21.15.132:0/1945484395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f419d0f0 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f8eee640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37f96ef640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 0x7f37f4198100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37f002c180 tx=0x7f37f00045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0041020 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f419d380 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f002c9f0 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0039040 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.299+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f41a4c70 con 0x7f37f41017e0 2024-03-18T23:20:36.301 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.300+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f4068610 con 0x7f37f41017e0 2024-03-18T23:20:36.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.301+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f37f0044020 con 0x7f37f41017e0 2024-03-18T23:20:36.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.302+0000 7f37ea7fc640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 0x7f37e00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.302+0000 7f37f8eee640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 0x7f37e00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.302 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.302+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f37f00f0860 con 0x7f37f41017e0 2024-03-18T23:20:36.303 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.302+0000 7f37f8eee640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 0x7f37e00a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37f40ff980 tx=0x7f37e4005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.309 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.310+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f37f004b020 con 0x7f37f41017e0 2024-03-18T23:20:36.487 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.488+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37f4109d50 con 0x7f37f41017e0 2024-03-18T23:20:36.488 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.489+0000 7f37ea7fc640 1 -- 172.21.15.132:0/1945484395 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23176 (secure 0 0 0) 0x7f37f00bd6d0 con 0x7f37f41017e0 2024-03-18T23:20:36.489 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:36.489 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":101,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:20:08.908133+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4032784803":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6800/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3928210624":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3187950145":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2898017683":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 msgr2=0x7f37e00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 0x7f37e00a4e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37f40ff980 tx=0x7f37e4005f90 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 msgr2=0x7f37f4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 0x7f37f4198100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37f002c180 tx=0x7f37f00045b0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 shutdown_connections 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f37e00a2970 0x7f37e00a4e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37f41077f0 0x7f37f419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f41021a0 0x7f37f4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.491+0000 7f37fb953640 1 --2- 172.21.15.132:0/1945484395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f41017e0 0x7f37f4198100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.492+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 >> 172.21.15.132:0/1945484395 conn(0x7f37f40fd290 msgr2=0x7f37f40fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:36.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.492+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 shutdown_connections 2024-03-18T23:20:36.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.492+0000 7f37fb953640 1 -- 172.21.15.132:0/1945484395 wait complete. 2024-03-18T23:20:36.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T23:02:45.295776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T23:07:12.208250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T23:07:13.086908+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T23:20:36.503 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 -- 172.21.15.132:0/3541539969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 msgr2=0x7f9380101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 --2- 172.21.15.132:0/3541539969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f9380101c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9374009cd0 tx=0x7f937402c0f0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 -- 172.21.15.132:0/3541539969 shutdown_connections 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 --2- 172.21.15.132:0/3541539969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380102170 0x7f938010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 --2- 172.21.15.132:0/3541539969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f9380101c30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 --2- 172.21.15.132:0/3541539969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93801077c0 0x7f9380107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 -- 172.21.15.132:0/3541539969 >> 172.21.15.132:0/3541539969 conn(0x7f9380078c20 msgr2=0x7f9380079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:36.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 -- 172.21.15.132:0/3541539969 shutdown_connections 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.609+0000 7f938761f640 1 -- 172.21.15.132:0/3541539969 wait complete. 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 Processor -- start 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 -- start start 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9380102170 0x7f9380198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93801077c0 0x7f938019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9380111b90 con 0x7f93801017b0 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9380111a10 con 0x7f93801077c0 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9380111d10 con 0x7f9380102170 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45122/0 (socket says 172.21.15.132:45122) 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 -- 172.21.15.132:0/163245322 learned_addr learned my addr 172.21.15.132:0/163245322 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f9384bba640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9380102170 0x7f9380198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f9385bbc640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93801077c0 0x7f938019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 -- 172.21.15.132:0/163245322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9380102170 msgr2=0x7f9380198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9380102170 0x7f9380198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 -- 172.21.15.132:0/163245322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93801077c0 msgr2=0x7f938019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93801077c0 0x7f938019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 -- 172.21.15.132:0/163245322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938019d0e0 con 0x7f93801017b0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f93853bb640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f937000e9f0 tx=0x7f937000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937000cd80 con 0x7f93801017b0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938019d3d0 con 0x7f93801017b0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9370010910 con 0x7f93801017b0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.610+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700188f0 con 0x7f93801017b0 2024-03-18T23:20:36.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.611+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801a4c60 con 0x7f93801017b0 2024-03-18T23:20:36.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.611+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9380068a60 con 0x7f93801017b0 2024-03-18T23:20:36.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.612+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f9370010ac0 con 0x7f93801017b0 2024-03-18T23:20:36.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.613+0000 7f937e7fc640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 0x7f93680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.613+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f93700cf560 con 0x7f93801017b0 2024-03-18T23:20:36.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.613+0000 7f9384bba640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 0x7f93680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.614+0000 7f9384bba640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 0x7f93680a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9374004af0 tx=0x7f9374005c50 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.616 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.616+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f937009bc70 con 0x7f93801017b0 2024-03-18T23:20:36.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.791+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9380199440 con 0x7f93801017b0 2024-03-18T23:20:36.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.791+0000 7f937e7fc640 1 -- 172.21.15.132:0/163245322 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f93700860a0 con 0x7f93801017b0 2024-03-18T23:20:36.791 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 1 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 msgr2=0x7f93680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 0x7f93680a4e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9374004af0 tx=0x7f9374005c50 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 msgr2=0x7f93801980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f937000e9f0 tx=0x7f937000eeb0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 shutdown_connections 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f93680a2970 0x7f93680a4e30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93801077c0 0x7f938019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9380102170 0x7f9380198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 --2- 172.21.15.132:0/163245322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93801017b0 0x7f93801980f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.793+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 >> 172.21.15.132:0/163245322 conn(0x7f9380078c20 msgr2=0x7f9380104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.794+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 shutdown_connections 2024-03-18T23:20:36.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.794+0000 7f938761f640 1 -- 172.21.15.132:0/163245322 wait complete. 2024-03-18T23:20:36.804 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T23:20:36.911 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 -- 172.21.15.132:0/1380718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041017b0 msgr2=0x7fa904101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 --2- 172.21.15.132:0/1380718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041017b0 0x7fa904101c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009930 tx=0x7fa8f402bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 -- 172.21.15.132:0/1380718031 shutdown_connections 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 --2- 172.21.15.132:0/1380718031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 0x7fa90410e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 --2- 172.21.15.132:0/1380718031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041017b0 0x7fa904101c30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 --2- 172.21.15.132:0/1380718031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041077c0 0x7fa904107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 -- 172.21.15.132:0/1380718031 >> 172.21.15.132:0/1380718031 conn(0x7fa904078c20 msgr2=0x7fa904079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 -- 172.21.15.132:0/1380718031 shutdown_connections 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.910+0000 7fa90b532640 1 -- 172.21.15.132:0/1380718031 wait complete. 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 Processor -- start 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 -- start start 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.912 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 0x7fa904198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041077c0 0x7fa90419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111b90 con 0x7fa9041017b0 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111a10 con 0x7fa904102170 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa90b532640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111d10 con 0x7fa9041077c0 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45134/0 (socket says 172.21.15.132:45134) 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 -- 172.21.15.132:0/301968299 learned_addr learned my addr 172.21.15.132:0/301968299 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa909acf640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041077c0 0x7fa90419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa908acd640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 0x7fa904198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 -- 172.21.15.132:0/301968299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041077c0 msgr2=0x7fa90419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041077c0 0x7fa90419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 -- 172.21.15.132:0/301968299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 msgr2=0x7fa904198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 0x7fa904198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:36.913 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.911+0000 7fa9092ce640 1 -- 172.21.15.132:0/301968299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90419d0e0 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa9092ce640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9f0 tx=0x7fa90000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90000ccc0 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90419d3d0 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9041a4c60 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa900010910 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.912+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900018880 con 0x7fa9041017b0 2024-03-18T23:20:36.914 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.913+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e8005290 con 0x7fa9041017b0 2024-03-18T23:20:36.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.914+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fa900010ac0 con 0x7fa9041017b0 2024-03-18T23:20:36.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.914+0000 7fa8fa7fc640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 0x7fa8f00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:36.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.914+0000 7fa908acd640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 0x7fa8f00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:36.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.914+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fa9000cf120 con 0x7fa9041017b0 2024-03-18T23:20:36.915 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.915+0000 7fa908acd640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 0x7fa8f00a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4005ab0 tx=0x7fa8f4005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:36.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:36.919+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa90009be80 con 0x7fa9041017b0 2024-03-18T23:20:37.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.090+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8e8005850 con 0x7fa9041017b0 2024-03-18T23:20:37.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.091+0000 7fa8fa7fc640 1 -- 172.21.15.132:0/301968299 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7fa90009b780 con 0x7fa9041017b0 2024-03-18T23:20:37.090 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 32 2024-03-18T23:20:37.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 msgr2=0x7fa8f00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 0x7fa8f00a4e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4005ab0 tx=0x7fa8f4005fd0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 msgr2=0x7fa9041980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9f0 tx=0x7fa90000edf0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 shutdown_connections 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fa8f00a2970 0x7fa8f00a4e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9041077c0 0x7fa90419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa904102170 0x7fa904198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 --2- 172.21.15.132:0/301968299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9041017b0 0x7fa9041980f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 >> 172.21.15.132:0/301968299 conn(0x7fa904078c20 msgr2=0x7fa9041041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 shutdown_connections 2024-03-18T23:20:37.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.093+0000 7fa90b532640 1 -- 172.21.15.132:0/301968299 wait complete. 2024-03-18T23:20:37.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T23:20:37.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 -- 172.21.15.132:0/2487621468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b00101460 msgr2=0x7f8b00101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 --2- 172.21.15.132:0/2487621468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b00101460 0x7f8b00101840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009900 tx=0x7f8af402bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 -- 172.21.15.132:0/2487621468 shutdown_connections 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 --2- 172.21.15.132:0/2487621468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00105ae0 0x7f8b0010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 --2- 172.21.15.132:0/2487621468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00101e10 0x7f8b001053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 --2- 172.21.15.132:0/2487621468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b00101460 0x7f8b00101840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 -- 172.21.15.132:0/2487621468 >> 172.21.15.132:0/2487621468 conn(0x7f8b00075a60 msgr2=0x7f8b00075e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 -- 172.21.15.132:0/2487621468 shutdown_connections 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 -- 172.21.15.132:0/2487621468 wait complete. 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.209+0000 7f8b05532640 1 Processor -- start 2024-03-18T23:20:37.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 -- start start 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105ae0 0x7f8b001985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 0x7f8b0019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8b00111c20 con 0x7f8b00101460 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b00111aa0 con 0x7f8b0019c960 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8b05532640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8b00111da0 con 0x7f8b00105ae0 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45144/0 (socket says 172.21.15.132:45144) 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 -- 172.21.15.132:0/1059443257 learned_addr learned my addr 172.21.15.132:0/1059443257 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8aff7fe640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 0x7f8b0019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afe7fc640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105ae0 0x7f8b001985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 -- 172.21.15.132:0/1059443257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105ae0 msgr2=0x7f8b001985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105ae0 0x7f8b001985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 -- 172.21.15.132:0/1059443257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 msgr2=0x7f8b0019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 0x7f8b0019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 -- 172.21.15.132:0/1059443257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0019d450 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8aff7fe640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 0x7f8b0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.210+0000 7f8afeffd640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006010 tx=0x7f8af402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.211+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4041070 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.211+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0019d6e0 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.211+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8af4004450 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.211+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4003e00 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.211+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b001a4f00 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.212+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b00068a60 con 0x7f8b00101460 2024-03-18T23:20:37.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.213+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f8af4044070 con 0x7f8b00101460 2024-03-18T23:20:37.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.213+0000 7f8ae7fff640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 0x7f8ae80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.213+0000 7f8afe7fc640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 0x7f8ae80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.213+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f8af40f07b0 con 0x7f8b00101460 2024-03-18T23:20:37.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.214+0000 7f8afe7fc640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 0x7f8ae80a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b000fd8a0 tx=0x7f8af0005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:37.217 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.218+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8af40bcfd0 con 0x7f8b00101460 2024-03-18T23:20:37.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:37 smithi132 ceph-mon[32020]: pgmap v353: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:37.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1945484395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:37.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/163245322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.364 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:37 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/301968299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.394 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.395+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b00108cd0 con 0x7f8b00101460 2024-03-18T23:20:37.394 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.395+0000 7f8ae7fff640 1 -- 172.21.15.132:0/1059443257 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f8af40c1070 con 0x7f8b00101460 2024-03-18T23:20:37.394 INFO:teuthology.orchestra.run.smithi132.stdout:pg_num: 64 2024-03-18T23:20:37.396 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 msgr2=0x7f8ae80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 0x7f8ae80a4e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b000fd8a0 tx=0x7f8af0005f90 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 msgr2=0x7f8b001980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006010 tx=0x7f8af402cb30 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 shutdown_connections 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f8ae80a2970 0x7f8ae80a4e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0019c960 0x7f8b0019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b00105ae0 0x7f8b001985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 --2- 172.21.15.132:0/1059443257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b00101460 0x7f8b001980a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 >> 172.21.15.132:0/1059443257 conn(0x7f8b00075a60 msgr2=0x7f8b000fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 shutdown_connections 2024-03-18T23:20:37.397 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.397+0000 7f8b05532640 1 -- 172.21.15.132:0/1059443257 wait complete. 2024-03-18T23:20:37.408 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 -- 172.21.15.132:0/2125420955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 msgr2=0x7f8a00100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 --2- 172.21.15.132:0/2125420955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00100b30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009930 tx=0x7f89f402bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 -- 172.21.15.132:0/2125420955 shutdown_connections 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 --2- 172.21.15.132:0/2125420955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a0010d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 --2- 172.21.15.132:0/2125420955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00100b30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 --2- 172.21.15.132:0/2125420955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 0x7f8a00106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 -- 172.21.15.132:0/2125420955 >> 172.21.15.132:0/2125420955 conn(0x7f8a000777f0 msgr2=0x7f8a00077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 -- 172.21.15.132:0/2125420955 shutdown_connections 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.511+0000 7f8a06827640 1 -- 172.21.15.132:0/2125420955 wait complete. 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 Processor -- start 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 -- start start 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 0x7f8a001a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00110ad0 con 0x7f8a00101070 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00110950 con 0x7f8a001066c0 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f8a06827640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00110c50 con 0x7f8a001006b0 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.512+0000 7f89ff7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45174/0 (socket says 172.21.15.132:45174) 2024-03-18T23:20:37.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 -- 172.21.15.132:0/3592406910 learned_addr learned my addr 172.21.15.132:0/3592406910 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89fffff640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f8a04dc4640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 0x7f8a001a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 -- 172.21.15.132:0/3592406910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 msgr2=0x7f8a00198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 -- 172.21.15.132:0/3592406910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 msgr2=0x7f8a001a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 0x7f8a001a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 -- 172.21.15.132:0/3592406910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001a4510 con 0x7f8a00101070 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89ff7fe640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009230 tx=0x7f89f402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4041070 con 0x7f8a00101070 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001a46e0 con 0x7f8a00101070 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f4004450 con 0x7f8a00101070 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4003de0 con 0x7f8a00101070 2024-03-18T23:20:37.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.513+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001a4ba0 con 0x7f8a00101070 2024-03-18T23:20:37.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.514+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00101df0 con 0x7f8a00101070 2024-03-18T23:20:37.516 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.515+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f89f4044070 con 0x7f8a00101070 2024-03-18T23:20:37.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.516+0000 7f89fd7fa640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 0x7f89e80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:37.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.516+0000 7f89fffff640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 0x7f89e80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:37.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.516+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f89f40f06a0 con 0x7f8a00101070 2024-03-18T23:20:37.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.516+0000 7f89fffff640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 0x7f89e80a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009b50 tx=0x7f89f0005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:37.519 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.520+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f89f40bd510 con 0x7f8a00101070 2024-03-18T23:20:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:37 smithi144 ceph-mon[66781]: pgmap v353: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1945484395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/163245322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:37 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/301968299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.706+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a0002d1f0 con 0x7f8a00101070 2024-03-18T23:20:37.705 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.706+0000 7f89fd7fa640 1 -- 172.21.15.132:0/3592406910 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+961 (secure 0 0 0) 0x7f89f40bce10 con 0x7f8a00101070 2024-03-18T23:20:37.705 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/4032784803 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/3730787943 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/3730787943 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/1441879282 2024-03-19T23:00:18.961831+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/3928210624 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/2712624534 2024-03-19T23:00:18.961831+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/3187950145 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/3371561746 2024-03-19T23:00:18.961831+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/2898017683 2024-03-19T23:09:15.212885+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6801/1223581697 2024-03-19T22:59:52.674103+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/2884383249 2024-03-19T22:59:52.674103+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/3371561746 2024-03-19T23:00:18.961831+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:6800/1223581697 2024-03-19T22:59:52.674103+0000 2024-03-18T23:20:37.706 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/4178269826 2024-03-19T22:59:52.674103+0000 2024-03-18T23:20:37.707 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/3720862041 2024-03-19T22:59:52.674103+0000 2024-03-18T23:20:37.707 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132:0/175028242 2024-03-19T23:00:18.961831+0000 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:listed 16 entries 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 msgr2=0x7f89e80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 0x7f89e80a4e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009b50 tx=0x7f89f0005f90 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 msgr2=0x7f8a00199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009230 tx=0x7f89f402cb30 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 shutdown_connections 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f89e80a2970 0x7f89e80a4e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a001066c0 0x7f8a001a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a00101070 0x7f8a00199490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 --2- 172.21.15.132:0/3592406910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a001006b0 0x7f8a00198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 >> 172.21.15.132:0/3592406910 conn(0x7f8a000777f0 msgr2=0x7f8a00102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 shutdown_connections 2024-03-18T23:20:37.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:37.708+0000 7f8a06827640 1 -- 172.21.15.132:0/3592406910 wait complete. 2024-03-18T23:20:37.718 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-18T23:20:37.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:20:37.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-18T23:20:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:37 smithi196 ceph-mon[67089]: pgmap v353: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-18T23:20:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1945484395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/163245322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:37 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/301968299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T23:20:37.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:20:37.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:20:37.764 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T23:20:37.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:20:37.828 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-03-18T23:20:37.842 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T23:20:37.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:20:37.843 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T23:20:37.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T23:20:37.911 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-03-18T23:20:38.015 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-18T23:20:38.027 INFO:tasks.cephadm:Teardown begin 2024-03-18T23:20:38.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T23:20:38.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1423482844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 msgr2=0x7fca1810e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1423482844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca1810e670 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fca10009900 tx=0x7fca1002bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1423482844 shutdown_connections 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1423482844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca1810e670 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1423482844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18069e70 0x7fca1806a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.132 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1423482844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18107820 0x7fca18107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1423482844 >> 172.21.15.132:0/1423482844 conn(0x7fca18076520 msgr2=0x7fca18076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1423482844 shutdown_connections 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1423482844 wait complete. 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.131+0000 7fca1d8f8640 1 Processor -- start 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 -- start start 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18069e70 0x7fca18100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18107820 0x7fca181015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fca18111c00 con 0x7fca1806a810 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca18111a80 con 0x7fca18069e70 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca1d8f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca18111d80 con 0x7fca18107820 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45196/0 (socket says 172.21.15.132:45196) 2024-03-18T23:20:38.133 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 -- 172.21.15.132:0/1386506473 learned_addr learned my addr 172.21.15.132:0/1386506473 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca16ffd640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18069e70 0x7fca18100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca177fe640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18107820 0x7fca181015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 -- 172.21.15.132:0/1386506473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18107820 msgr2=0x7fca181015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18107820 0x7fca181015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 -- 172.21.15.132:0/1386506473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18069e70 msgr2=0x7fca18100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18069e70 0x7fca18100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 -- 172.21.15.132:0/1386506473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca181a65c0 con 0x7fca1806a810 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.132+0000 7fca167fc640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e9f0 tx=0x7fca0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.133+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00cd80 con 0x7fca1806a810 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.133+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c005500 con 0x7fca1806a810 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.133+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca181a68b0 con 0x7fca1806a810 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.133+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c010690 con 0x7fca1806a810 2024-03-18T23:20:38.134 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.133+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca181a6db0 con 0x7fca1806a810 2024-03-18T23:20:38.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.134+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca18102ee0 con 0x7fca1806a810 2024-03-18T23:20:38.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.135+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fca0c0046c0 con 0x7fca1806a810 2024-03-18T23:20:38.135 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.135+0000 7fca1c8f6640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 0x7fca000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.135+0000 7fca16ffd640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 0x7fca000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.135+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fca0c0cf260 con 0x7fca1806a810 2024-03-18T23:20:38.136 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.136+0000 7fca16ffd640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 0x7fca000a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca08005fd0 tx=0x7fca08009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.138 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.140+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca0c09bfc0 con 0x7fca1806a810 2024-03-18T23:20:38.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.353+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca1802d1f0 con 0x7fca1806a810 2024-03-18T23:20:38.352 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.353+0000 7fca1c8f6640 1 -- 172.21.15.132:0/1386506473 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11888 (secure 0 0 0) 0x7fca0c09b8c0 con 0x7fca1806a810 2024-03-18T23:20:38.353 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:38.353 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:20:00.890668+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24485,24493]},"id":1}]} 2024-03-18T23:20:38.353 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-03-18T23:20:38.355 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 msgr2=0x7fca000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 0x7fca000a4e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca08005fd0 tx=0x7fca08009500 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 msgr2=0x7fca18101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e9f0 tx=0x7fca0c00edf0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 shutdown_connections 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fca000a2970 0x7fca000a4e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca18107820 0x7fca181015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca1806a810 0x7fca18101080 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 --2- 172.21.15.132:0/1386506473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca18069e70 0x7fca18100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.355+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 >> 172.21.15.132:0/1386506473 conn(0x7fca18076520 msgr2=0x7fca180fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.356+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 shutdown_connections 2024-03-18T23:20:38.356 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.356+0000 7fca1d8f8640 1 -- 172.21.15.132:0/1386506473 wait complete. 2024-03-18T23:20:38.366 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T23:20:38.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- 172.21.15.132:0/714087633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 msgr2=0x7fdb00106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 --2- 172.21.15.132:0/714087633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb00106b10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009930 tx=0x7fdaf002bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:38.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- 172.21.15.132:0/714087633 shutdown_connections 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 --2- 172.21.15.132:0/714087633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 0x7fdb0010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 --2- 172.21.15.132:0/714087633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 --2- 172.21.15.132:0/714087633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb00106b10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- 172.21.15.132:0/714087633 >> 172.21.15.132:0/714087633 conn(0x7fdb00077820 msgr2=0x7fdb00077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- 172.21.15.132:0/714087633 shutdown_connections 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- 172.21.15.132:0/714087633 wait complete. 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 Processor -- start 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.470+0000 7fdb05129640 1 -- start start 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 0x7fdb001994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00110b00 con 0x7fdb00106730 2024-03-18T23:20:38.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00110980 con 0x7fdb00100720 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00110c80 con 0x7fdb001010e0 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45210/0 (socket says 172.21.15.132:45210) 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 -- 172.21.15.132:0/3119006156 learned_addr learned my addr 172.21.15.132:0/3119006156 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdafed9d640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaf7fff640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 0x7fdb001994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 -- 172.21.15.132:0/3119006156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 msgr2=0x7fdb001994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 0x7fdb001994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 -- 172.21.15.132:0/3119006156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 msgr2=0x7fdb00198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 -- 172.21.15.132:0/3119006156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb001a4540 con 0x7fdb00106730 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdafed9d640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdaff59e640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eb10 tx=0x7fdaf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8004b20 con 0x7fdb00106730 2024-03-18T23:20:38.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001a4710 con 0x7fdb00106730 2024-03-18T23:20:38.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf8004cc0 con 0x7fdb00106730 2024-03-18T23:20:38.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.471+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8010760 con 0x7fdb00106730 2024-03-18T23:20:38.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.472+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001a4c60 con 0x7fdb00106730 2024-03-18T23:20:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1059443257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:38.474 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:38 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3592406910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:20:38.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.473+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb00068610 con 0x7fdb00106730 2024-03-18T23:20:38.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.474+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fdaf8005020 con 0x7fdb00106730 2024-03-18T23:20:38.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.474+0000 7fdafcd99640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 0x7fdaec0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.474+0000 7fdafed9d640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 0x7fdaec0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.475+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fdaf80cf370 con 0x7fdb00106730 2024-03-18T23:20:38.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.475+0000 7fdafed9d640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 0x7fdaec0a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005ab0 tx=0x7fdaf0005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.479+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdaf809c0d0 con 0x7fdb00106730 2024-03-18T23:20:38.607 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1059443257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:38.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:38 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3592406910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:20:38.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1059443257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T23:20:38.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:38 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3592406910' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T23:20:38.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.691+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb00108c90 con 0x7fdb00106730 2024-03-18T23:20:38.691 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.692+0000 7fdafcd99640 1 -- 172.21.15.132:0/3119006156 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11888 (secure 0 0 0) 0x7fdaf809b9d0 con 0x7fdb00106730 2024-03-18T23:20:38.691 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:38.691 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1067641512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1067641512},{"type":"v1","addr":"172.21.15.132:6835","nonce":1067641512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2263462076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2263462076},{"type":"v1","addr":"172.21.15.132:6837","nonce":2263462076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/4158656425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":4158656425},{"type":"v1","addr":"172.21.15.132:6839","nonce":4158656425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6841/3314453651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6840","nonce":3314453651},{"type":"v1","addr":"172.21.15.132:6841","nonce":3314453651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3738348032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3738348032},{"type":"v1","addr":"172.21.15.196:6833","nonce":3738348032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/2374016108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2374016108},{"type":"v1","addr":"172.21.15.144:6833","nonce":2374016108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T23:07:13.853025+0000","modified":"2024-03-18T23:20:00.890668+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/769602526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":769602526},{"type":"v1","addr":"172.21.15.144:6835","nonce":769602526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/559973533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":559973533},{"type":"v1","addr":"172.21.15.196:6835","nonce":559973533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/3483317240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3483317240},{"type":"v1","addr":"172.21.15.196:6837","nonce":3483317240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/4044803735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":4044803735},{"type":"v1","addr":"172.21.15.144:6837","nonce":4044803735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.196:6839/1134284387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":1134284387},{"type":"v1","addr":"172.21.15.196:6839","nonce":1134284387}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6839/837888145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":837888145},{"type":"v1","addr":"172.21.15.144:6839","nonce":837888145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24485,24493]},"id":1}]} 2024-03-18T23:20:38.691 INFO:teuthology.orchestra.run.smithi132.stderr:dumped fsmap epoch 25 2024-03-18T23:20:38.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 msgr2=0x7fdaec0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 0x7fdaec0a4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005ab0 tx=0x7fdaf0005fd0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 msgr2=0x7fdb001a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.693 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eb10 tx=0x7fdaf800c2d0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 shutdown_connections 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fdaec0a2970 0x7fdaec0a4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb00106730 0x7fdb001a4000 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb001010e0 0x7fdb001994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 --2- 172.21.15.132:0/3119006156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb00100720 0x7fdb00198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 >> 172.21.15.132:0/3119006156 conn(0x7fdb00077820 msgr2=0x7fdb00103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 shutdown_connections 2024-03-18T23:20:38.694 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.694+0000 7fdb05129640 1 -- 172.21.15.132:0/3119006156 wait complete. 2024-03-18T23:20:38.705 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 -- 172.21.15.132:0/3918995302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2778069e70 msgr2=0x7f277806a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 --2- 172.21.15.132:0/3918995302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2778069e70 0x7f277806a2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2764009890 tx=0x7f276402bb70 comp rx=0 tx=0).stop 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 -- 172.21.15.132:0/3918995302 shutdown_connections 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 --2- 172.21.15.132:0/3918995302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f277806a7f0 0x7f277810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 --2- 172.21.15.132:0/3918995302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2778069e70 0x7f277806a2b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 --2- 172.21.15.132:0/3918995302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27781077b0 0x7f2778107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 -- 172.21.15.132:0/3918995302 >> 172.21.15.132:0/3918995302 conn(0x7f2778076520 msgr2=0x7f2778076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 -- 172.21.15.132:0/3918995302 shutdown_connections 2024-03-18T23:20:38.810 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.808+0000 7f277f205640 1 -- 172.21.15.132:0/3918995302 wait complete. 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 Processor -- start 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 -- start start 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 0x7f27781980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 0x7f277819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2778111b80 con 0x7f277806a7f0 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2778111a00 con 0x7f27781077b0 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277f205640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2778111d00 con 0x7f2778069e70 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f2777fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f2777fff640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45232/0 (socket says 172.21.15.132:45232) 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f2777fff640 1 -- 172.21.15.132:0/579943075 learned_addr learned my addr 172.21.15.132:0/579943075 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277cfa1640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 0x7f27781980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.809+0000 7f277d7a2640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 0x7f277819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.811 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 -- 172.21.15.132:0/579943075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 msgr2=0x7f27781980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 0x7f27781980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 -- 172.21.15.132:0/579943075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 msgr2=0x7f277819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 0x7f277819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 -- 172.21.15.132:0/579943075 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277819d0d0 con 0x7f277806a7f0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f277cfa1640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 0x7f27781980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f277d7a2640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 0x7f277819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2777fff640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2764000c00 tx=0x7f2764004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764041020 con 0x7f277806a7f0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277819d3c0 con 0x7f277806a7f0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276402c9f0 con 0x7f277806a7f0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764039040 con 0x7f277806a7f0 2024-03-18T23:20:38.812 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.810+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a4c50 con 0x7f277806a7f0 2024-03-18T23:20:38.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.811+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2778102ee0 con 0x7f277806a7f0 2024-03-18T23:20:38.813 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.812+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f2764044020 con 0x7f277806a7f0 2024-03-18T23:20:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.813+0000 7f2775ffb640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 0x7f27680a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.813+0000 7f277cfa1640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 0x7f27680a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.813+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f27640f0550 con 0x7f277806a7f0 2024-03-18T23:20:38.814 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.813+0000 7f277cfa1640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 0x7f27680a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f276c0096c0 tx=0x7f276c009260 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:38.816 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.817+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f276404b020 con 0x7f277806a7f0 2024-03-18T23:20:38.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.996+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2778199430 con 0x7f277806a7f0 2024-03-18T23:20:38.996 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.997+0000 7f2775ffb640 1 -- 172.21.15.132:0/579943075 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23176 (secure 0 0 0) 0x7f27640bd3c0 con 0x7f277806a7f0 2024-03-18T23:20:38.997 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-03-18T23:20:38.997 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":101,"fsid":"0a41095a-e57b-11ee-95c9-87774f69a715","created":"2024-03-18T22:59:19.051809+0000","modified":"2024-03-18T23:20:08.908133+0000","last_up_change":"2024-03-18T23:06:25.199949+0000","last_in_change":"2024-03-18T23:06:03.924347+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T23:02:45.295776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T23:07:12.208250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T23:07:13.086908+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f8845c7f-34df-49a6-96ee-d71d153dc0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6803","nonce":3523914432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6805","nonce":3523914432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6809","nonce":3523914432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":3523914432},{"type":"v1","addr":"172.21.15.132:6807","nonce":3523914432}]},"public_addr":"172.21.15.132:6803/3523914432","cluster_addr":"172.21.15.132:6805/3523914432","heartbeat_back_addr":"172.21.15.132:6809/3523914432","heartbeat_front_addr":"172.21.15.132:6807/3523914432","state":["exists","up"]},{"osd":1,"uuid":"3e2cb994-9178-41e5-9724-62360d566915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6801","nonce":1173848453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6803","nonce":1173848453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6807","nonce":1173848453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1173848453},{"type":"v1","addr":"172.21.15.144:6805","nonce":1173848453}]},"public_addr":"172.21.15.144:6801/1173848453","cluster_addr":"172.21.15.144:6803/1173848453","heartbeat_back_addr":"172.21.15.144:6807/1173848453","heartbeat_front_addr":"172.21.15.144:6805/1173848453","state":["exists","up"]},{"osd":2,"uuid":"2adeb4b3-32ca-446e-9801-d535dd74c2d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6801","nonce":1879525213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6803","nonce":1879525213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6807","nonce":1879525213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1879525213},{"type":"v1","addr":"172.21.15.196:6805","nonce":1879525213}]},"public_addr":"172.21.15.196:6801/1879525213","cluster_addr":"172.21.15.196:6803/1879525213","heartbeat_back_addr":"172.21.15.196:6807/1879525213","heartbeat_front_addr":"172.21.15.196:6805/1879525213","state":["exists","up"]},{"osd":3,"uuid":"253c2374-3879-4485-8ecb-a893a210b624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6811","nonce":3584795430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6813","nonce":3584795430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6817","nonce":3584795430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3584795430},{"type":"v1","addr":"172.21.15.132:6815","nonce":3584795430}]},"public_addr":"172.21.15.132:6811/3584795430","cluster_addr":"172.21.15.132:6813/3584795430","heartbeat_back_addr":"172.21.15.132:6817/3584795430","heartbeat_front_addr":"172.21.15.132:6815/3584795430","state":["exists","up"]},{"osd":4,"uuid":"5e8564c1-d17a-453d-a5e2-af4740fb0d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6809","nonce":962231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6811","nonce":962231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6815","nonce":962231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":962231389},{"type":"v1","addr":"172.21.15.144:6813","nonce":962231389}]},"public_addr":"172.21.15.144:6809/962231389","cluster_addr":"172.21.15.144:6811/962231389","heartbeat_back_addr":"172.21.15.144:6815/962231389","heartbeat_front_addr":"172.21.15.144:6813/962231389","state":["exists","up"]},{"osd":5,"uuid":"5eedd04e-128b-491d-82e6-1988d13b1145","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6809","nonce":1616570087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6811","nonce":1616570087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6815","nonce":1616570087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1616570087},{"type":"v1","addr":"172.21.15.196:6813","nonce":1616570087}]},"public_addr":"172.21.15.196:6809/1616570087","cluster_addr":"172.21.15.196:6811/1616570087","heartbeat_back_addr":"172.21.15.196:6815/1616570087","heartbeat_front_addr":"172.21.15.196:6813/1616570087","state":["exists","up"]},{"osd":6,"uuid":"df2ea776-34b0-45ca-8bfa-f7e5428ce0c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6819","nonce":428682768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6821","nonce":428682768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6825","nonce":428682768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":428682768},{"type":"v1","addr":"172.21.15.132:6823","nonce":428682768}]},"public_addr":"172.21.15.132:6819/428682768","cluster_addr":"172.21.15.132:6821/428682768","heartbeat_back_addr":"172.21.15.132:6825/428682768","heartbeat_front_addr":"172.21.15.132:6823/428682768","state":["exists","up"]},{"osd":7,"uuid":"cf357f13-d06f-49c0-b598-1cadcd305fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6817","nonce":3972442848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6819","nonce":3972442848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6823","nonce":3972442848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3972442848},{"type":"v1","addr":"172.21.15.144:6821","nonce":3972442848}]},"public_addr":"172.21.15.144:6817/3972442848","cluster_addr":"172.21.15.144:6819/3972442848","heartbeat_back_addr":"172.21.15.144:6823/3972442848","heartbeat_front_addr":"172.21.15.144:6821/3972442848","state":["exists","up"]},{"osd":8,"uuid":"a8d245b3-4a54-4056-92f6-0c30db4759cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6817","nonce":3559751523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6819","nonce":3559751523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6823","nonce":3559751523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3559751523},{"type":"v1","addr":"172.21.15.196:6821","nonce":3559751523}]},"public_addr":"172.21.15.196:6817/3559751523","cluster_addr":"172.21.15.196:6819/3559751523","heartbeat_back_addr":"172.21.15.196:6823/3559751523","heartbeat_front_addr":"172.21.15.196:6821/3559751523","state":["exists","up"]},{"osd":9,"uuid":"5b2f5b52-8981-48f2-9229-27493b6d62f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6827","nonce":118980095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6829","nonce":118980095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6833","nonce":118980095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":118980095},{"type":"v1","addr":"172.21.15.132:6831","nonce":118980095}]},"public_addr":"172.21.15.132:6827/118980095","cluster_addr":"172.21.15.132:6829/118980095","heartbeat_back_addr":"172.21.15.132:6833/118980095","heartbeat_front_addr":"172.21.15.132:6831/118980095","state":["exists","up"]},{"osd":10,"uuid":"5922d4cd-937c-44ec-b82c-7fec2f16c8c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6825","nonce":2873915844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6827","nonce":2873915844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6831","nonce":2873915844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2873915844},{"type":"v1","addr":"172.21.15.144:6829","nonce":2873915844}]},"public_addr":"172.21.15.144:6825/2873915844","cluster_addr":"172.21.15.144:6827/2873915844","heartbeat_back_addr":"172.21.15.144:6831/2873915844","heartbeat_front_addr":"172.21.15.144:6829/2873915844","state":["exists","up"]},{"osd":11,"uuid":"d26113fc-541e-4090-89e8-909b4c0d5d05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6825","nonce":946882152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6827","nonce":946882152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6831","nonce":946882152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":946882152},{"type":"v1","addr":"172.21.15.196:6829","nonce":946882152}]},"public_addr":"172.21.15.196:6825/946882152","cluster_addr":"172.21.15.196:6827/946882152","heartbeat_back_addr":"172.21.15.196:6831/946882152","heartbeat_front_addr":"172.21.15.196:6829/946882152","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:01:54.761360+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:18.304258+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:02:41.614268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:06.091214+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:30.134529+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:03:53.784581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:18.193548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:04:42.927155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:07.293404+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:32.478021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:05:57.212002+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T23:06:22.230657+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:0/4032784803":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6800/3730787943":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/1441879282":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3928210624":"2024-03-19T23:09:15.212885+0000","172.21.15.132:0/2712624534":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/3187950145":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:0/2898017683":"2024-03-19T23:09:15.212885+0000","172.21.15.132:6801/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/2884383249":"2024-03-19T22:59:52.674103+0000","172.21.15.132:6800/3371561746":"2024-03-19T23:00:18.961831+0000","172.21.15.132:6800/1223581697":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/4178269826":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/3720862041":"2024-03-19T22:59:52.674103+0000","172.21.15.132:0/175028242":"2024-03-19T23:00:18.961831+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 msgr2=0x7f27680a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 0x7f27680a4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f276c0096c0 tx=0x7f276c009260 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 msgr2=0x7f2778198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2764000c00 tx=0x7f2764004550 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 shutdown_connections 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f27680a2970 0x7f27680a4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27781077b0 0x7f277819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f277806a7f0 0x7f2778198620 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 --2- 172.21.15.132:0/579943075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2778069e70 0x7f27781980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 >> 172.21.15.132:0/579943075 conn(0x7f2778076520 msgr2=0x7f27780fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 shutdown_connections 2024-03-18T23:20:38.999 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:38.999+0000 7f277f205640 1 -- 172.21.15.132:0/579943075 wait complete. 2024-03-18T23:20:39.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 -- 172.21.15.132:0/76934748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5684101070 msgr2=0x7f568410d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 --2- 172.21.15.132:0/76934748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5684101070 0x7f568410d580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f567c009cd0 tx=0x7f567c02c0f0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 -- 172.21.15.132:0/76934748 shutdown_connections 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 --2- 172.21.15.132:0/76934748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5684101070 0x7f568410d580 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 --2- 172.21.15.132:0/76934748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56841006b0 0x7f5684100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 --2- 172.21.15.132:0/76934748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f5684106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 -- 172.21.15.132:0/76934748 >> 172.21.15.132:0/76934748 conn(0x7f56840777f0 msgr2=0x7f5684077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:39.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 -- 172.21.15.132:0/76934748 shutdown_connections 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.114+0000 7f5688c81640 1 -- 172.21.15.132:0/76934748 wait complete. 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 Processor -- start 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 -- start start 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5684101070 0x7f5684198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f568419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5684110ad0 con 0x7f56841006b0 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5684110950 con 0x7f5684101070 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5688c81640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5684110c50 con 0x7f56841066c0 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45254/0 (socket says 172.21.15.132:45254) 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 -- 172.21.15.132:0/633353853 learned_addr learned my addr 172.21.15.132:0/633353853 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:39.117 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5681d9b640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5684101070 0x7f5684198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5682d9d640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f568419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 -- 172.21.15.132:0/633353853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 msgr2=0x7f568419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f568419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 -- 172.21.15.132:0/633353853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5684101070 msgr2=0x7f5684198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5684101070 0x7f5684198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 -- 172.21.15.132:0/633353853 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568419cfe0 con 0x7f56841006b0 2024-03-18T23:20:39.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f5682d9d640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f568419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.115+0000 7f568259c640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f566c00e9f0 tx=0x7f566c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.116+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00cd80 con 0x7f56841006b0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.116+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568419d2d0 con 0x7f56841006b0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.116+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c010910 con 0x7f56841006b0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.116+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0188f0 con 0x7f56841006b0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.116+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841a4b60 con 0x7f56841006b0 2024-03-18T23:20:39.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.117+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5684101df0 con 0x7f56841006b0 2024-03-18T23:20:39.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.118+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f566c010ac0 con 0x7f56841006b0 2024-03-18T23:20:39.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.118+0000 7f567b7fe640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 0x7f56700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.118+0000 7f5681d9b640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 0x7f56700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.118+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f566c0cf240 con 0x7f56841006b0 2024-03-18T23:20:39.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.119+0000 7f5681d9b640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 0x7f56700a4e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5684100230 tx=0x7f5674005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:39.121 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.123+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f566c09bfa0 con 0x7f56841006b0 2024-03-18T23:20:39.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.334+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5684109c60 con 0x7f56841006b0 2024-03-18T23:20:39.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.336+0000 7f567b7fe640 1 -- 172.21.15.132:0/633353853 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f566c09b8a0 con 0x7f56841006b0 2024-03-18T23:20:39.335 INFO:teuthology.orchestra.run.smithi132.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T23:20:39.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 msgr2=0x7f56700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 0x7f56700a4e30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5684100230 tx=0x7f5674005f90 comp rx=0 tx=0).stop 2024-03-18T23:20:39.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 msgr2=0x7f5684197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f566c00e9f0 tx=0x7f566c00eeb0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 shutdown_connections 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f56700a2970 0x7f56700a4e30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56841066c0 0x7f568419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5684101070 0x7f5684198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 --2- 172.21.15.132:0/633353853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56841006b0 0x7f5684197ff0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 >> 172.21.15.132:0/633353853 conn(0x7f56840777f0 msgr2=0x7f568406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 shutdown_connections 2024-03-18T23:20:39.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.339+0000 7f5688c81640 1 -- 172.21.15.132:0/633353853 wait complete. 2024-03-18T23:20:39.350 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 -- 172.21.15.132:0/2722585954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24100740 msgr2=0x7fbc24100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 --2- 172.21.15.132:0/2722585954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24100740 0x7fbc24100bc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009930 tx=0x7fbc1402bc30 comp rx=0 tx=0).stop 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 -- 172.21.15.132:0/2722585954 shutdown_connections 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 --2- 172.21.15.132:0/2722585954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24101100 0x7fbc2410d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 --2- 172.21.15.132:0/2722585954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24100740 0x7fbc24100bc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 --2- 172.21.15.132:0/2722585954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24106750 0x7fbc24106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 -- 172.21.15.132:0/2722585954 >> 172.21.15.132:0/2722585954 conn(0x7fbc24077880 msgr2=0x7fbc24077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 -- 172.21.15.132:0/2722585954 shutdown_connections 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.455+0000 7fbc2b885640 1 -- 172.21.15.132:0/2722585954 wait complete. 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 Processor -- start 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 -- start start 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 0x7fbc241980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 0x7fbc2419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24111c60 con 0x7fbc24101100 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24111ae0 con 0x7fbc24106750 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc2b885640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24111de0 con 0x7fbc24100740 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc28e20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc29621640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 0x7fbc241980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc28e20640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45270/0 (socket says 172.21.15.132:45270) 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc28e20640 1 -- 172.21.15.132:0/1426322929 learned_addr learned my addr 172.21.15.132:0/1426322929 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.456+0000 7fbc29e22640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 0x7fbc2419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 -- 172.21.15.132:0/1426322929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 msgr2=0x7fbc241980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 0x7fbc241980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 -- 172.21.15.132:0/1426322929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 msgr2=0x7fbc2419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 0x7fbc2419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 -- 172.21.15.132:0/1426322929 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2419d090 con 0x7fbc24101100 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc29621640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 0x7fbc241980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc29e22640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 0x7fbc2419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc28e20640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc14004f50 tx=0x7fbc1402ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14041070 con 0x7fbc24101100 2024-03-18T23:20:39.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc140044d0 con 0x7fbc24101100 2024-03-18T23:20:39.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2419d320 con 0x7fbc24101100 2024-03-18T23:20:39.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14034440 con 0x7fbc24101100 2024-03-18T23:20:39.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.457+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc241a4c10 con 0x7fbc24101100 2024-03-18T23:20:39.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.458+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc24068a60 con 0x7fbc24101100 2024-03-18T23:20:39.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.459+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7fbc14044070 con 0x7fbc24101100 2024-03-18T23:20:39.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.459+0000 7fbc1a7fc640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 0x7fbc100a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:39.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.460+0000 7fbc29621640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 0x7fbc100a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:39.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.460+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8727+0+0 (secure 0 0 0) 0x7fbc140f0970 con 0x7fbc24101100 2024-03-18T23:20:39.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.460+0000 7fbc29621640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 0x7fbc100a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc200058c0 tx=0x7fbc2000a250 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:39.463 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.464+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc140bd0c0 con 0x7fbc24101100 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mon[66781]: pgmap v354: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1386506473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3119006156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/579943075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mds[97504]: -- 172.21.15.144:0/1965281998 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622698f7a40 con 0x5622698ca800 2024-03-18T23:20:39.608 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: starting mds.e at 2024-03-18T23:20:39.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mds[97817]: -- 172.21.15.144:0/1996114706 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a469297a40 con 0x55a46926a800 2024-03-18T23:20:39.609 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: starting mds.h at 2024-03-18T23:20:39.609 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.610 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.610 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-mds[98128]: -- 172.21.15.144:0/2640270503 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d38675a40 con 0x562d38648800 2024-03-18T23:20:39.610 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:20:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: starting mds.b at 2024-03-18T23:20:39.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:39 smithi132 ceph-mon[32020]: pgmap v354: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:39.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:39 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1386506473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:39 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3119006156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:39 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/579943075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a[32016]: 2024-03-18T23:20:39.335+0000 7fda9e17e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:20:39.678 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:39.677+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbc24108cb0 con 0x7fbc24101100 2024-03-18T23:20:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mon[67089]: pgmap v354: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T23:20:39.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1386506473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3119006156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/579943075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mds[97409]: -- 172.21.15.196:0/372075578 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562aaebbfa40 con 0x562aaeb92800 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: starting mds.f at 2024-03-18T23:20:39.724 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mds[98026]: -- 172.21.15.196:0/1403642102 <== mon.0 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589fb03da40 con 0x5589fb011000 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: starting mds.i at 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: ignoring --setuser ceph since I am not root 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: ignoring --setgroup ceph since I am not root 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-mds[97720]: -- 172.21.15.196:0/3792033670 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d7978fa40 con 0x559d79762c00 2024-03-18T23:20:39.725 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:20:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: starting mds.c at 2024-03-18T23:20:40.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.341+0000 7fbc1a7fc640 1 -- 172.21.15.132:0/1426322929 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fbc140c1160 con 0x7fbc24101100 2024-03-18T23:20:40.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 msgr2=0x7fbc100a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:40.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 0x7fbc100a4e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc200058c0 tx=0x7fbc2000a250 comp rx=0 tx=0).stop 2024-03-18T23:20:40.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 msgr2=0x7fbc241985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:40.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc14004f50 tx=0x7fbc1402ce20 comp rx=0 tx=0).stop 2024-03-18T23:20:40.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 shutdown_connections 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7fbc100a2970 0x7fbc100a4e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc24106750 0x7fbc2419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc24101100 0x7fbc241985e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 --2- 172.21.15.132:0/1426322929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc24100740 0x7fbc241980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 >> 172.21.15.132:0/1426322929 conn(0x7fbc24077880 msgr2=0x7fbc241032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 shutdown_connections 2024-03-18T23:20:40.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.345+0000 7fbc2b885640 1 -- 172.21.15.132:0/1426322929 wait complete. 2024-03-18T23:20:40.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T23:20:40.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 -- 172.21.15.132:0/3321519013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 msgr2=0x7f488810d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:40.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 --2- 172.21.15.132:0/3321519013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f488810d580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4880009cd0 tx=0x7f488002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 -- 172.21.15.132:0/3321519013 shutdown_connections 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 --2- 172.21.15.132:0/3321519013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f488810d580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 --2- 172.21.15.132:0/3321519013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 0x7f4888100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 --2- 172.21.15.132:0/3321519013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 0x7f4888106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 -- 172.21.15.132:0/3321519013 >> 172.21.15.132:0/3321519013 conn(0x7f48880777f0 msgr2=0x7f4888077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 -- 172.21.15.132:0/3321519013 shutdown_connections 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.472+0000 7f488e17e640 1 -- 172.21.15.132:0/3321519013 wait complete. 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 Processor -- start 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 -- start start 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 0x7f48881980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:40.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 0x7f488819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110ad0 con 0x7f4888101070 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110950 con 0x7f48881006b0 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f488e17e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110c50 con 0x7f48881066c0 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f4886ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f4886ffd640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45290/0 (socket says 172.21.15.132:45290) 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.473+0000 7f4886ffd640 1 -- 172.21.15.132:0/3089671789 learned_addr learned my addr 172.21.15.132:0/3089671789 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4887fff640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 0x7f488819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f48877fe640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 0x7f48881980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 -- 172.21.15.132:0/3089671789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 msgr2=0x7f488819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 0x7f488819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 -- 172.21.15.132:0/3089671789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 msgr2=0x7f48881980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 0x7f48881980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 -- 172.21.15.132:0/3089671789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488819d0c0 con 0x7f4888101070 2024-03-18T23:20:40.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4886ffd640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c890 tx=0x7f487c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c004220 con 0x7f4888101070 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c0043c0 con 0x7f4888101070 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488819d3b0 con 0x7f4888101070 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c004a00 con 0x7f4888101070 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.474+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a4c40 con 0x7f4888101070 2024-03-18T23:20:40.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.475+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888101df0 con 0x7f4888101070 2024-03-18T23:20:40.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.476+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f487c0079b0 con 0x7f4888101070 2024-03-18T23:20:40.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.477+0000 7f4884ff9640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 0x7f48700a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:40.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.477+0000 7f48877fe640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 0x7f48700a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:40.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.477+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8727+0+0 (secure 0 0 0) 0x7f487c0cecb0 con 0x7f4888101070 2024-03-18T23:20:40.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.477+0000 7f48877fe640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 0x7f48700a4e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4878005fd0 tx=0x7f487800b510 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:40.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.482+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f487c014030 con 0x7f4888101070 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-18T23:20:40.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:40 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-18T23:20:40.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:40 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:20:40.658 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:40.659+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48881096a0 con 0x7f4888101070 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/633353853' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-18T23:20:40.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:40 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T23:20:41.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.342+0000 7f4884ff9640 1 -- 172.21.15.132:0/3089671789 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f487c09b940 con 0x7f4888101070 2024-03-18T23:20:41.342 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_metadata' removed 2024-03-18T23:20:41.344 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.345+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 msgr2=0x7f48700a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.345+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 0x7f48700a4e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4878005fd0 tx=0x7f487800b510 comp rx=0 tx=0).stop 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.345+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 msgr2=0x7f4888198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.345+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c890 tx=0x7f487c00cd50 comp rx=0 tx=0).stop 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 shutdown_connections 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f48700a2970 0x7f48700a4e30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48881066c0 0x7f488819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4888101070 0x7f4888198610 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.345 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 --2- 172.21.15.132:0/3089671789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48881006b0 0x7f48881980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 >> 172.21.15.132:0/3089671789 conn(0x7f48880777f0 msgr2=0x7f488806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:41.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 shutdown_connections 2024-03-18T23:20:41.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.346+0000 7f488e17e640 1 -- 172.21.15.132:0/3089671789 wait complete. 2024-03-18T23:20:41.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 -- 172.21.15.132:0/1326782313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 msgr2=0x7f57b8110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1326782313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b8110890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b002c0f0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 -- 172.21.15.132:0/1326782313 shutdown_connections 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1326782313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b8110890 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1326782313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 0x7f57b8100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1326782313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 -- 172.21.15.132:0/1326782313 >> 172.21.15.132:0/1326782313 conn(0x7f57b8076ce0 msgr2=0x7f57b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 -- 172.21.15.132:0/1326782313 shutdown_connections 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.472+0000 7f57c06ea640 1 -- 172.21.15.132:0/1326782313 wait complete. 2024-03-18T23:20:41.474 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 Processor -- start 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 -- start start 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 0x7f57b819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113e10 con 0x7f57b810e4a0 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113c90 con 0x7f57b81027d0 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57c06ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113f90 con 0x7f57b8102c40 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:45312/0 (socket says 172.21.15.132:45312) 2024-03-18T23:20:41.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 -- 172.21.15.132:0/1209901928 learned_addr learned my addr 172.21.15.132:0/1209901928 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57be486640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bdc85640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 0x7f57b819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 -- 172.21.15.132:0/1209901928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 msgr2=0x7f57b819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 0x7f57b819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 -- 172.21.15.132:0/1209901928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 msgr2=0x7f57b819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 -- 172.21.15.132:0/1209901928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b819f280 con 0x7f57b810e4a0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57be486640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57bec87640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004af0 tx=0x7f57b0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.473+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0042020 con 0x7f57b810e4a0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.474+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b00346c0 con 0x7f57b810e4a0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.474+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b819f510 con 0x7f57b810e4a0 2024-03-18T23:20:41.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.474+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0039040 con 0x7f57b810e4a0 2024-03-18T23:20:41.477 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.474+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a6e00 con 0x7f57b810e4a0 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.475+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b8068a60 con 0x7f57b810e4a0 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.476+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 25) ==== 137422+0+0 (secure 0 0 0) 0x7f57b0045020 con 0x7f57b810e4a0 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.476+0000 7f57af7fe640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 0x7f57a40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.476+0000 7f57be486640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 0x7f57a40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.476+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8202+0+0 (secure 0 0 0) 0x7f57b00f0de0 con 0x7f57b810e4a0 2024-03-18T23:20:41.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.477+0000 7f57be486640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 0x7f57a40a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57b40058c0 tx=0x7f57b4005850 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T23:20:41.480 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.481+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57b004c020 con 0x7f57b810e4a0 2024-03-18T23:20:41.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: pgmap v356: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-03-18T23:20:41.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:20:41.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:20:41.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: Cluster is now healthy 2024-03-18T23:20:41.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6836/3053665938,v1:172.21.15.144:6838/3053665938] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6838/2469067570,v1:172.21.15.196:6839/2469067570] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6834/2902510879,v1:172.21.15.144:6835/2902510879] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6834/2566555035,v1:172.21.15.196:6835/2566555035] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.144:6837/4255440948,v1:172.21.15.144:6839/4255440948] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: mds.? [v2:172.21.15.196:6836/3662591453,v1:172.21.15.196:6837/3662591453] up:boot 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: fsmap 12 up:standby 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:20:41.609 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:20:41.610 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:41 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: pgmap v356: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: Cluster is now healthy 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:20:41.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6836/3053665938,v1:172.21.15.144:6838/3053665938] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6838/2469067570,v1:172.21.15.196:6839/2469067570] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6834/2902510879,v1:172.21.15.144:6835/2902510879] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6834/2566555035,v1:172.21.15.196:6835/2566555035] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.144:6837/4255440948,v1:172.21.15.144:6839/4255440948] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: mds.? [v2:172.21.15.196:6836/3662591453,v1:172.21.15.196:6837/3662591453] up:boot 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: fsmap 12 up:standby 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:20:41.615 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:41 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:41.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:41.658+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f57b81a6fd0 con 0x7f57b810e4a0 2024-03-18T23:20:41.725 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: pgmap v356: 97 pgs: 97 active+clean; 5.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: Cluster is now healthy 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1426322929' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6836/3053665938,v1:172.21.15.144:6838/3053665938] up:boot 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6838/2469067570,v1:172.21.15.196:6839/2469067570] up:boot 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6834/2902510879,v1:172.21.15.144:6835/2902510879] up:boot 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6834/2566555035,v1:172.21.15.196:6835/2566555035] up:boot 2024-03-18T23:20:41.726 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.144:6837/4255440948,v1:172.21.15.144:6839/4255440948] up:boot 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: mds.? [v2:172.21.15.196:6836/3662591453,v1:172.21.15.196:6837/3662591453] up:boot 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: fsmap 12 up:standby 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='mgr.15294 172.21.15.132:0/3066418424' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T23:20:41.727 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:41 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:42.347 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.348+0000 7f57af7fe640 1 -- 172.21.15.132:0/1209901928 <== mon.0 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f57b00bdd90 con 0x7f57b810e4a0 2024-03-18T23:20:42.347 INFO:teuthology.orchestra.run.smithi132.stderr:pool 'cephfs_data' removed 2024-03-18T23:20:42.350 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.350+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 msgr2=0x7f57a40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.350+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 0x7f57a40a4e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57b40058c0 tx=0x7f57b4005850 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.350+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 msgr2=0x7f57b819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.350+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f57b0004af0 tx=0x7f57b0032f40 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 shutdown_connections 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:6800/4165175733,v1:172.21.15.132:6801/4165175733] conn(0x7f57a40a2970 0x7f57a40a4e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b810e4a0 0x7f57b819eb50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b8102c40 0x7f57b819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 --2- 172.21.15.132:0/1209901928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b81027d0 0x7f57b819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 >> 172.21.15.132:0/1209901928 conn(0x7f57b8076ce0 msgr2=0x7f57b810f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 shutdown_connections 2024-03-18T23:20:42.351 INFO:teuthology.orchestra.run.smithi132.stderr:2024-03-18T23:20:42.351+0000 7f57c06ea640 1 -- 172.21.15.132:0/1209901928 wait complete. 2024-03-18T23:20:42.366 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:20:42.395 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:20:42.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:20:42.465 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-18T23:20:42.465 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:20:42.481 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:20:42.499 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T23:20:42.521 INFO:tasks.cephadm:Stopping all daemons... 2024-03-18T23:20:42.522 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-18T23:20:42.522 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a 2024-03-18T23:20:42.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:42 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:20:42.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:42 smithi144 ceph-mon[66781]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:20:42.608 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:42 smithi144 ceph-mon[66781]: from='client.? 172.21.15.132:0/1209901928' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:42.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:20:42.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 ceph-mon[32020]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:20:42.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 ceph-mon[32020]: from='client.? 172.21.15.132:0/1209901928' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:42.614 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 systemd[1]: Stopping Ceph mon.a for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:42.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:42 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/3089671789' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T23:20:42.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:42 smithi196 ceph-mon[67089]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T23:20:42.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:42 smithi196 ceph-mon[67089]: from='client.? 172.21.15.132:0/1209901928' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T23:20:43.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a[32016]: 2024-03-18T23:20:42.815+0000 7fdaa3989700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:43.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a[32016]: 2024-03-18T23:20:42.815+0000 7fdaa3989700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-18T23:20:43.114 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:42 smithi132 podman[91869]: 2024-03-18 23:20:42.847315974 +0000 UTC m=+0.131094125 container died b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-18T23:20:43.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:43 smithi132 podman[91869]: 2024-03-18 23:20:43.216334595 +0000 UTC m=+0.500112753 container cleanup b748b36d26819cb5e26942e247fc240352e797bb2b2d7bdb26e9ff9822b7cf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-18T23:20:43.500 INFO:journalctl@ceph.mon.a.smithi132.stdout:Mar 18 23:20:43 smithi132 bash[91869]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-a 2024-03-18T23:20:43.641 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.a.service' 2024-03-18T23:20:43.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:43.687 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-18T23:20:43.687 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-18T23:20:43.687 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.b 2024-03-18T23:20:43.992 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:43 smithi144 systemd[1]: Stopping Ceph mon.b for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:44.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-b[66777]: 2024-03-18T23:20:43.992+0000 7f0f12c0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:44.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-b[66777]: 2024-03-18T23:20:43.992+0000 7f0f12c0e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-18T23:20:44.358 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:44 smithi144 podman[103572]: 2024-03-18 23:20:44.146621157 +0000 UTC m=+0.266009468 container died b5996aff448e7e1c1d10c3d43d86ed760599ee6bcfece39b35e7271c5b834be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:20:44.727 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:44 smithi144 podman[103572]: 2024-03-18 23:20:44.485111847 +0000 UTC m=+0.604500205 container cleanup b5996aff448e7e1c1d10c3d43d86ed760599ee6bcfece39b35e7271c5b834be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-b, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:20:44.727 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 18 23:20:44 smithi144 bash[103572]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-b 2024-03-18T23:20:44.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.b.service' 2024-03-18T23:20:44.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:44.892 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-18T23:20:44.892 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-18T23:20:44.893 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.c 2024-03-18T23:20:45.191 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:44 smithi196 systemd[1]: Stopping Ceph mon.c for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c[67084]: 2024-03-18T23:20:45.191+0000 7f75f5ebb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c[67084]: 2024-03-18T23:20:45.191+0000 7f75f5ebb700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-18T23:20:45.473 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:45 smithi196 podman[104225]: 2024-03-18 23:20:45.2233816 +0000 UTC m=+0.132865641 container died 54ceaf63f9e536b6d80e81936e6aa7836252478fa117404025e47f130d055d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:45.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:45 smithi196 podman[104225]: 2024-03-18 23:20:45.567416126 +0000 UTC m=+0.476900184 container cleanup 54ceaf63f9e536b6d80e81936e6aa7836252478fa117404025e47f130d055d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-18T23:20:45.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Mar 18 23:20:45 smithi196 bash[104225]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mon-c 2024-03-18T23:20:45.942 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mon.c.service' 2024-03-18T23:20:45.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:45.991 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-18T23:20:45.991 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-18T23:20:45.992 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x 2024-03-18T23:20:46.311 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 systemd[1]: Stopping Ceph mgr.x for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:46.592 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:20:46.312+0000 7fb4cc814700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:46.592 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x[32253]: 2024-03-18T23:20:46.312+0000 7fb4cc814700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-18T23:20:46.592 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 podman[91989]: 2024-03-18 23:20:46.343794233 +0000 UTC m=+0.161176119 container died a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-18T23:20:46.864 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 podman[91989]: 2024-03-18 23:20:46.746882265 +0000 UTC m=+0.564264152 container cleanup a033519837ae73f12cdfe41e252bbd7a3c118add047bc74fe01a2573526c0c6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:46.864 INFO:journalctl@ceph.mgr.x.smithi132.stdout:Mar 18 23:20:46 smithi132 bash[91989]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-x 2024-03-18T23:20:47.106 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-18T23:20:47.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:47.147 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-18T23:20:47.147 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-18T23:20:47.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.y 2024-03-18T23:20:47.492 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:20:47 smithi144 systemd[1]: Stopping Ceph mgr.y for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:47.772 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:20:47 smithi144 podman[103696]: 2024-03-18 23:20:47.493805141 +0000 UTC m=+0.157927627 container died 9825a78361bfd2d703deb4db5070d6323e8c7b1c70d73f8dfab5f0f3d770fde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T23:20:48.108 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:20:47 smithi144 podman[103696]: 2024-03-18 23:20:47.856807552 +0000 UTC m=+0.520930035 container cleanup 9825a78361bfd2d703deb4db5070d6323e8c7b1c70d73f8dfab5f0f3d770fde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:48.108 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 18 23:20:47 smithi144 bash[103696]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-y 2024-03-18T23:20:48.284 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-18T23:20:48.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:48.321 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-18T23:20:48.321 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-18T23:20:48.321 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.z 2024-03-18T23:20:48.641 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:48 smithi196 systemd[1]: Stopping Ceph mgr.z for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:48.973 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:48 smithi196 podman[104350]: 2024-03-18 23:20:48.642550352 +0000 UTC m=+0.125379366 container died 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:49.329 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:48 smithi196 podman[104350]: 2024-03-18 23:20:48.987978648 +0000 UTC m=+0.470807678 container cleanup 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-18T23:20:49.329 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:48 smithi196 bash[104350]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z 2024-03-18T23:20:49.329 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:49 smithi196 podman[104364]: 2024-03-18 23:20:49.130824792 +0000 UTC m=+0.484152398 container remove 6268e0ea3f0e548a71cfbd97d95473ac4d722df29e7109f37f94104b96526109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-18T23:20:49.329 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Mar 18 23:20:49 smithi196 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-18T23:20:49.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-18T23:20:49.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:49.380 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-18T23:20:49.380 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-18T23:20:49.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.0 2024-03-18T23:20:49.692 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:49 smithi132 systemd[1]: Stopping Ceph osd.0 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:50.114 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0[46793]: 2024-03-18T23:20:49.692+0000 7f564823f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:50.114 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0[46793]: 2024-03-18T23:20:49.692+0000 7f564823f700 -1 osd.0 104 *** Got signal Terminated *** 2024-03-18T23:20:50.114 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0[46793]: 2024-03-18T23:20:49.692+0000 7f564823f700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:20:55.364 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:55 smithi132 podman[92120]: 2024-03-18 23:20:55.07852868 +0000 UTC m=+5.499721091 container died a5510b1f5087e9871aa41093c65ed7ba2e56292483765dd01bae22e7a34194d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-18T23:20:55.701 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:55 smithi132 podman[92120]: 2024-03-18 23:20:55.459430362 +0000 UTC m=+5.880622772 container cleanup a5510b1f5087e9871aa41093c65ed7ba2e56292483765dd01bae22e7a34194d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:20:55.701 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:55 smithi132 bash[92120]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0 2024-03-18T23:20:56.022 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:55 smithi132 podman[92137]: 2024-03-18 23:20:55.702555231 +0000 UTC m=+0.626174121 container remove a5510b1f5087e9871aa41093c65ed7ba2e56292483765dd01bae22e7a34194d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-18T23:20:56.321 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:55.927796785 +0000 UTC m=+0.018152687 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:20:56.321 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:56.036588702 +0000 UTC m=+0.126944588 container create 3e0b45b81da008687cc647a3345a2e9a1a6c6e1f6330b3db502632009c7c8513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T23:20:56.321 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:56.230922514 +0000 UTC m=+0.321278402 container init 3e0b45b81da008687cc647a3345a2e9a1a6c6e1f6330b3db502632009c7c8513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:20:56.321 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:56.233987999 +0000 UTC m=+0.324343887 container start 3e0b45b81da008687cc647a3345a2e9a1a6c6e1f6330b3db502632009c7c8513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-18T23:20:56.321 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:56.322627024 +0000 UTC m=+0.412982932 container attach 3e0b45b81da008687cc647a3345a2e9a1a6c6e1f6330b3db502632009c7c8513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:20:56.614 INFO:journalctl@ceph.osd.0.smithi132.stdout:Mar 18 23:20:56 smithi132 podman[92228]: 2024-03-18 23:20:56.534122103 +0000 UTC m=+0.624477983 container died 3e0b45b81da008687cc647a3345a2e9a1a6c6e1f6330b3db502632009c7c8513 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-18T23:20:57.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.0.service' 2024-03-18T23:20:57.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:20:57.140 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-18T23:20:57.140 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-18T23:20:57.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.1 2024-03-18T23:20:57.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:20:57 smithi144 systemd[1]: Stopping Ceph osd.1 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:20:57.694 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:20:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1[71357]: 2024-03-18T23:20:57.434+0000 7f0b0f1ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:20:57.694 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:20:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1[71357]: 2024-03-18T23:20:57.434+0000 7f0b0f1ee700 -1 osd.1 104 *** Got signal Terminated *** 2024-03-18T23:20:57.694 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:20:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1[71357]: 2024-03-18T23:20:57.434+0000 7f0b0f1ee700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:03.108 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:02 smithi144 podman[103818]: 2024-03-18 23:21:02.835952915 +0000 UTC m=+5.508267950 container died 5f02ad44985e2ea275db84ebeb48e35163a48a63ff482c0d2ac41bf02be03d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-18T23:21:03.509 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:03 smithi144 podman[103818]: 2024-03-18 23:21:03.209544739 +0000 UTC m=+5.881859745 container cleanup 5f02ad44985e2ea275db84ebeb48e35163a48a63ff482c0d2ac41bf02be03d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-18T23:21:03.509 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:03 smithi144 bash[103818]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1 2024-03-18T23:21:03.827 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:03 smithi144 podman[103847]: 2024-03-18 23:21:03.511014904 +0000 UTC m=+0.671921548 container remove 5f02ad44985e2ea275db84ebeb48e35163a48a63ff482c0d2ac41bf02be03d94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:04.108 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:03 smithi144 podman[103937]: 2024-03-18 23:21:03.732328978 +0000 UTC m=+0.018317047 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:04.108 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:03 smithi144 podman[103937]: 2024-03-18 23:21:03.836924553 +0000 UTC m=+0.122912659 container create 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-18T23:21:04.108 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:04 smithi144 podman[103937]: 2024-03-18 23:21:04.073128786 +0000 UTC m=+0.359116859 container init 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T23:21:04.108 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:04 smithi144 podman[103937]: 2024-03-18 23:21:04.076238094 +0000 UTC m=+0.362226154 container start 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T23:21:04.371 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:04 smithi144 podman[103937]: 2024-03-18 23:21:04.164864311 +0000 UTC m=+0.450852372 container attach 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-18T23:21:04.667 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:04 smithi144 podman[103937]: 2024-03-18 23:21:04.373315625 +0000 UTC m=+0.659303677 container died 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-18T23:21:04.919 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 18 23:21:04 smithi144 podman[103964]: 2024-03-18 23:21:04.918315243 +0000 UTC m=+0.539071608 container remove 8fd4589565a10628c26a49366874c8078635100087f06e24c94991f6556b46b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:04.938 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.1.service' 2024-03-18T23:21:04.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:04.980 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-18T23:21:04.980 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-18T23:21:04.980 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.2 2024-03-18T23:21:05.282 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:05 smithi196 systemd[1]: Stopping Ceph osd.2 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:05.722 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:21:05.282+0000 7f36efde9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:05.723 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:21:05.282+0000 7f36efde9700 -1 osd.2 104 *** Got signal Terminated *** 2024-03-18T23:21:05.723 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2[72399]: 2024-03-18T23:21:05.282+0000 7f36efde9700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:10.963 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:10 smithi196 podman[104486]: 2024-03-18 23:21:10.693344042 +0000 UTC m=+5.511065695 container died 53c78363265b407372861b6a2f55b4d329341f605c7166ad88c896b58b604923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:11.222 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:10 smithi196 podman[104486]: 2024-03-18 23:21:10.965183967 +0000 UTC m=+5.782905612 container cleanup 53c78363265b407372861b6a2f55b4d329341f605c7166ad88c896b58b604923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-03-18T23:21:11.222 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:10 smithi196 bash[104486]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2 2024-03-18T23:21:11.223 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104501]: 2024-03-18 23:21:11.199708984 +0000 UTC m=+0.505579987 container remove 53c78363265b407372861b6a2f55b4d329341f605c7166ad88c896b58b604923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-18T23:21:11.779 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104597]: 2024-03-18 23:21:11.432656526 +0000 UTC m=+0.017622692 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:11.780 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104597]: 2024-03-18 23:21:11.542279954 +0000 UTC m=+0.127246103 container create 750cbcf8b05c1ab0ea3052fa9845c654ba8d4f32f8d871f7ccb3d863e7966f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-18T23:21:11.780 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104597]: 2024-03-18 23:21:11.778325128 +0000 UTC m=+0.363291288 container init 750cbcf8b05c1ab0ea3052fa9845c654ba8d4f32f8d871f7ccb3d863e7966f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:11.780 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104597]: 2024-03-18 23:21:11.781351618 +0000 UTC m=+0.366317780 container start 750cbcf8b05c1ab0ea3052fa9845c654ba8d4f32f8d871f7ccb3d863e7966f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5) 2024-03-18T23:21:12.069 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:11 smithi196 podman[104597]: 2024-03-18 23:21:11.878471029 +0000 UTC m=+0.463437195 container attach 750cbcf8b05c1ab0ea3052fa9845c654ba8d4f32f8d871f7ccb3d863e7966f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:12.336 INFO:journalctl@ceph.osd.2.smithi196.stdout:Mar 18 23:21:12 smithi196 podman[104597]: 2024-03-18 23:21:12.071082326 +0000 UTC m=+0.656048492 container died 750cbcf8b05c1ab0ea3052fa9845c654ba8d4f32f8d871f7ccb3d863e7966f2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-18T23:21:12.673 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.2.service' 2024-03-18T23:21:12.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:12.715 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-18T23:21:12.715 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-18T23:21:12.715 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.3 2024-03-18T23:21:13.011 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:12 smithi132 systemd[1]: Stopping Ceph osd.3 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:13.364 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:21:13.011+0000 7fc6189da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:13.364 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:21:13.011+0000 7fc6189da700 -1 osd.3 104 *** Got signal Terminated *** 2024-03-18T23:21:13.364 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:13 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:21:13.011+0000 7fc6189da700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:16.614 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:16 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:21:16.227+0000 7fc614fce700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:53.123318+0000 front 2024-03-18T23:20:53.123353+0000 (oldest deadline 2024-03-18T23:21:16.023524+0000) 2024-03-18T23:21:16.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:16 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:16.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:17.183 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:17 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:17.094+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:17.614 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:17 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3[53319]: 2024-03-18T23:21:17.184+0000 7fc614fce700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:53.123318+0000 front 2024-03-18T23:20:53.123353+0000 (oldest deadline 2024-03-18T23:21:16.023524+0000) 2024-03-18T23:21:17.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:17 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:17.390+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:17.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:17.661+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:18.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:18.086+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:18.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:18.118+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:18.697 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:18 smithi132 podman[92337]: 2024-03-18 23:21:18.550312808 +0000 UTC m=+5.636457021 container died 8533e7dacd3141e88dda57c8e182fa30772bc22c8ab7a45698297d7a8860f52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-18T23:21:18.698 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:18 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:18.423+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:18.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:18 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:18.360+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:18.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:18.633+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:19.044 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:18 smithi132 podman[92337]: 2024-03-18 23:21:18.870140931 +0000 UTC m=+5.956285149 container cleanup 8533e7dacd3141e88dda57c8e182fa30772bc22c8ab7a45698297d7a8860f52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-18T23:21:19.044 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:18 smithi132 bash[92337]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3 2024-03-18T23:21:19.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:19.045+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:19.364 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92366]: 2024-03-18 23:21:19.071321698 +0000 UTC m=+0.520296407 container remove 8533e7dacd3141e88dda57c8e182fa30772bc22c8ab7a45698297d7a8860f52b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-18T23:21:19.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:19.096+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:19.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:19 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:19.318+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:19.724 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:19 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:19.452+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:19.724 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92455]: 2024-03-18 23:21:19.385251776 +0000 UTC m=+0.019499962 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:19.724 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92455]: 2024-03-18 23:21:19.497209401 +0000 UTC m=+0.131457588 container create c00df3016584c4eb4a93afd3e6f06e7f6f2aea3b89a32114e76f8992cb166bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-18T23:21:19.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:19.624+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:20.010 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92455]: 2024-03-18 23:21:19.725635703 +0000 UTC m=+0.359883890 container init c00df3016584c4eb4a93afd3e6f06e7f6f2aea3b89a32114e76f8992cb166bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:20.010 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92455]: 2024-03-18 23:21:19.728982064 +0000 UTC m=+0.363230258 container start c00df3016584c4eb4a93afd3e6f06e7f6f2aea3b89a32114e76f8992cb166bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:20.010 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:19 smithi132 podman[92455]: 2024-03-18 23:21:19.808182463 +0000 UTC m=+0.442430650 container attach c00df3016584c4eb4a93afd3e6f06e7f6f2aea3b89a32114e76f8992cb166bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:20.301 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:20 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:20.029+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:20.301 INFO:journalctl@ceph.osd.3.smithi132.stdout:Mar 18 23:21:20 smithi132 podman[92455]: 2024-03-18 23:21:20.012175075 +0000 UTC m=+0.646423269 container died c00df3016584c4eb4a93afd3e6f06e7f6f2aea3b89a32114e76f8992cb166bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:20.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:20.103+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:20.576 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:20 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:20.483+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:20.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.3.service' 2024-03-18T23:21:20.607 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:20.353+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:20.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:20.644 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-18T23:21:20.644 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-18T23:21:20.644 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.4 2024-03-18T23:21:20.959 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:20.658+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:20.960 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:20 smithi144 systemd[1]: Stopping Ceph osd.4 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:20.960 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:20.961+0000 7f8e29d08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:20.960 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:20.961+0000 7f8e29d08700 -1 osd.4 104 *** Got signal Terminated *** 2024-03-18T23:21:20.960 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:20 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:20.961+0000 7f8e29d08700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:20.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:20.637+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:21.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:21 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:21.070+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:21.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:21 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:21.152+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:21.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:21 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:21.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:21.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:21 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:21.504+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:21.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:21 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:21.613+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:22.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:21 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:21.671+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:22.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:22 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:22.077+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:22.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:22 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:22.141+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:22.607 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:22 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:22.268+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:22.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:22 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:22.499+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:22.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:22 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:22.629+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:23.108 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:22 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:22.659+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:23.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:23 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:23.092+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:23.474 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:23 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:23.126+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:23.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:23 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:23.290+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:23.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:23 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:23.541+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:23.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:23 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:23.678+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:24.108 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:23 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:23.688+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:24.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:24 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:24.053+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:24.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:24 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:24.053+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:24.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:24 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:24.086+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:24.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:24 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:24.389+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:24.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:24 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:24.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:24.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:24 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:24.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:24.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:24 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:24.560+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:24.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:24 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:24.652+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:24.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:24 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:24.630+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:25.108 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:24 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:24.647+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:25.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:25 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:25.035+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:25.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:25 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:25.035+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:25.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:25 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:25.115+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:25.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:25 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:25.115+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:25.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:25 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:25.356+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:25.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:25 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:25.292+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:25.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:25 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:25.292+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:25.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:25 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:25.534+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:25.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:25 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:25.534+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:25.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:25 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:25.598+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:25.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:25 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:25.674+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:26.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:25 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4[78360]: 2024-03-18T23:21:25.619+0000 7f8e262fc700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.153127+0000 front 2024-03-18T23:20:54.153208+0000 (oldest deadline 2024-03-18T23:21:20.053134+0000) 2024-03-18T23:21:26.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:26.043+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:26.365 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:26.043+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:26.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:26 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:26.082+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:26.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:26 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:26.082+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:26.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:26 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:26.367+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:26.608 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:26 smithi144 podman[104045]: 2024-03-18 23:21:26.334093751 +0000 UTC m=+5.490421803 container died 5078d3deb68a97004f99525ac7ed615b4bc63ee978658df4d959ae7e93b55f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:21:26.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:26 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:26.311+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:26.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:26 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:26.311+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:26.747 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:26 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:26.609+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:26.747 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:26 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:26.681+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:26.748 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:26 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:26.681+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:26.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:26.581+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:26.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:26 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:26.581+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:26.944 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:26 smithi144 podman[104045]: 2024-03-18 23:21:26.744622556 +0000 UTC m=+5.900950607 container cleanup 5078d3deb68a97004f99525ac7ed615b4bc63ee978658df4d959ae7e93b55f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-18T23:21:26.944 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:26 smithi144 bash[104045]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4 2024-03-18T23:21:27.275 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:26 smithi144 podman[104063]: 2024-03-18 23:21:26.945705225 +0000 UTC m=+0.601603807 container remove 5078d3deb68a97004f99525ac7ed615b4bc63ee978658df4d959ae7e93b55f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T23:21:27.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:27.028+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:27.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:27.028+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:27.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:27 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:27.053+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:27.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:27 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:27.053+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:27.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.179331435 +0000 UTC m=+0.017432854 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:27.548 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.304881556 +0000 UTC m=+0.142982967 container create a3efbbceb6b3d03958bec1fb93c4c7de62be936b298e38d3794b9dfdb738589b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:27.548 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:27 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:27.402+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:27.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:27 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:27.298+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:27.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:27 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:27.298+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:27.842 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.549310586 +0000 UTC m=+0.387412002 container init a3efbbceb6b3d03958bec1fb93c4c7de62be936b298e38d3794b9dfdb738589b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:27.843 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.55291505 +0000 UTC m=+0.391016463 container start a3efbbceb6b3d03958bec1fb93c4c7de62be936b298e38d3794b9dfdb738589b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:21:27.843 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.632405354 +0000 UTC m=+0.470506758 container attach a3efbbceb6b3d03958bec1fb93c4c7de62be936b298e38d3794b9dfdb738589b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-18T23:21:27.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:27.564+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:27.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:27.564+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:27.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:27 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:27.642+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:27.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:27 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:27.700+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:27.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:27 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:27.700+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:28.107 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 18 23:21:27 smithi144 podman[104156]: 2024-03-18 23:21:27.844342114 +0000 UTC m=+0.682443529 container died a3efbbceb6b3d03958bec1fb93c4c7de62be936b298e38d3794b9dfdb738589b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True) 2024-03-18T23:21:28.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:28 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:28.077+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:28.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:28 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:28.077+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:28.409 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:28 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:28.345+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:28.409 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:28 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:28.345+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:28.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.4.service' 2024-03-18T23:21:28.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:28.040+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:28.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:28.040+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:28.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:28.477 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-18T23:21:28.477 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-18T23:21:28.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.5 2024-03-18T23:21:28.691 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:28 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:28.436+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:28.808 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:28.687+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:28.809 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 systemd[1]: Stopping Ceph osd.5 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:28.809 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:28.678+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:28.809 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:28.678+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:28.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:28 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:28.563+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:28.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:28 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:28.563+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:29.059 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:29.059+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:29.059 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:28.809+0000 7fd741739700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:29.059 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:28.809+0000 7fd741739700 -1 osd.5 104 *** Got signal Terminated *** 2024-03-18T23:21:29.059 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:28 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:28.809+0000 7fd741739700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:29.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:29.108+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:29.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:29.108+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:29.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:29.059+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:29.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:29 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:29.420+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:29.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:29 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:29.303+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:29.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:29 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:29.303+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:29.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:29.600+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:29.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:29 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:29.600+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:29.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:29.727+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:29.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:29.631+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:29.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:29 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:29.631+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:30.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:30.079+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:30.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:30.079+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:30.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:30.081+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:30.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:30.081+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:30.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:30 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:30.445+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:30.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:30 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:30.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:30.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:30 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:30.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:30.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:30.584+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:30.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:30 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:30.584+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:30.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:30.751+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:30.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:30.635+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:30.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:30 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:30.635+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:31.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:31 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:31.054+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:31.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:31 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:31.054+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:31.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:31.049+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:31.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:31.049+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:31.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:31 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:31.404+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:31.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:31 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:31.269+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:31.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:31 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:31.269+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:31.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:31.636+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:31.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:31.636+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:31.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:31.734+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:31.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:31 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:31.734+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:32.036 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:31 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:31.618+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:32.036 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:31 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:31.618+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:32.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:32.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:32.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:32.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:32.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:32.095+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:32.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:32.095+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:32.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:32.095+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:32.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:32.357+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:32.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:32.308+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:32.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:32.308+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:32.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:32 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:32.308+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:32.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:32.609+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:32.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:32.609+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:32.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:32.609+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:32.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:32.634+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:32.972 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:32.634+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:32.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:32.765+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:32.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:32 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:32.765+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:33.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:33.074+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:33.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:33.074+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:33.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:33.074+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:33.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:33.057+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:33.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:33.057+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:33.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:33.057+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:33.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:33.349+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:33.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:33.309+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:33.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:33.309+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:33.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:33 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:33.309+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:33.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:33.727+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:33.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:33.727+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:33.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:33.590+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:51.917931+0000 front 2024-03-18T23:20:51.917949+0000 (oldest deadline 2024-03-18T23:21:17.217919+0000) 2024-03-18T23:21:33.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:33.590+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.919003+0000 front 2024-03-18T23:21:01.919264+0000 (oldest deadline 2024-03-18T23:21:26.619074+0000) 2024-03-18T23:21:33.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:33 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5[78929]: 2024-03-18T23:21:33.590+0000 7fd739d25700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:07.720032+0000 front 2024-03-18T23:21:07.720060+0000 (oldest deadline 2024-03-18T23:21:33.020052+0000) 2024-03-18T23:21:34.103 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:33.618+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:34.103 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:33.618+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:34.103 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:33 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:33.618+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:34.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:34.104+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:34.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:34.104+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:34.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:34.104+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:34.473 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:34 smithi196 podman[104726]: 2024-03-18 23:21:34.332033437 +0000 UTC m=+5.673071463 container died 1e700265a99f8f23012cb55fa749fdcceb77506b4bc3885d331f4e1635844715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:34.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:34.090+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:34.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:34.090+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:34.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:34.090+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:34.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:34 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:34.310+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:34.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:34 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:34.349+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:34.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:34 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:34.349+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:34.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:34 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:34.349+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:34.959 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:34 smithi196 podman[104726]: 2024-03-18 23:21:34.691843411 +0000 UTC m=+6.032881500 container cleanup 1e700265a99f8f23012cb55fa749fdcceb77506b4bc3885d331f4e1635844715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-18T23:21:34.959 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:34 smithi196 bash[104726]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5 2024-03-18T23:21:34.959 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:34.710+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:34.959 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:34 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:34.710+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:35.074 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:34.665+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:35.075 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:34.665+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:35.075 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:34 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:34.665+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:35.222 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:34 smithi196 podman[104744]: 2024-03-18 23:21:34.960880809 +0000 UTC m=+0.626222905 container remove 1e700265a99f8f23012cb55fa749fdcceb77506b4bc3885d331f4e1635844715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:35.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:35.088+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:35.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:35.088+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:35.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:35.088+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:35.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:35.076+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:35.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:35.076+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:35.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:35.076+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:35.571 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.205031973 +0000 UTC m=+0.018002108 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:35.571 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.335346664 +0000 UTC m=+0.148316789 container create 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T23:21:35.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:35.292+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:35.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:35.292+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:35.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:35.342+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:35.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:35.342+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:35.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:35 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:35.342+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:35.859 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:35.676+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:35.860 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:35 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:35.676+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:35.860 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.571409382 +0000 UTC m=+0.384379525 container init 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:35.860 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.574679996 +0000 UTC m=+0.387650122 container start 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T23:21:35.860 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.654652879 +0000 UTC m=+0.467623019 container attach 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-18T23:21:36.048 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:35.668+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:36.048 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:35.668+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:36.048 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:35 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:35.668+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:36.125 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:35 smithi196 podman[104831]: 2024-03-18 23:21:35.861408878 +0000 UTC m=+0.674378997 container died 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:36.359 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.5.service' 2024-03-18T23:21:36.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:36.049+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:36.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:36.049+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:36.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:36.049+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:36 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:36.126+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:36 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:36.126+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:36 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:36.126+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:36 smithi196 podman[104858]: 2024-03-18 23:21:36.341429478 +0000 UTC m=+0.474854784 container remove 28487e4a324e54faa80d8253695308d11f8190a2c36e73b219b98ecf14439f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:36 smithi196 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:36 smithi196 systemd[1]: Stopped Ceph osd.5 for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:21:36.386 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 18 23:21:36 smithi196 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.5.service: Consumed 14.100s CPU time. 2024-03-18T23:21:36.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:36.401 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-18T23:21:36.401 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-18T23:21:36.401 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.6 2024-03-18T23:21:36.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:36.290+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:36.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:36.290+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:36.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:36.361+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:36.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:36.361+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:36.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:36 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:36.361+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:36.693 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 systemd[1]: Stopping Ceph osd.6 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:36.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:36 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:36.685+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:36.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:36 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:36.685+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.694+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.694+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.694+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.696+0000 7f833c9f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.696+0000 7f833c9f7700 -1 osd.6 104 *** Got signal Terminated *** 2024-03-18T23:21:37.037 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:36.696+0000 7f833c9f7700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:37.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:37.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:37.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:37.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:37.365 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:37.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:37.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:37 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:37.169+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:37.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:37 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:37.169+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:37.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:37 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:37.169+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:37.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:37.282+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:37.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:37.282+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:37.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:37.386+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:37.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:37.386+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:37.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:37 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:37.386+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:37.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:37 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:37.714+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:37.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:37 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:37.714+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:38.007 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:37.725+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:38.007 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:37.725+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:38.007 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:37 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:37.725+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:38.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:38.009+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:38.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:38.009+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:38.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:38.009+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:38.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:38 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:38.183+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:38.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:38 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:38.183+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:38.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:38 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:38.183+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:38.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:38.313+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:38.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:38.313+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:38.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:38.343+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:38.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:38.343+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:38.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:38 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:38.343+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:39.033 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:38.699+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:39.033 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:38.699+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:39.033 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:38.699+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:39.033 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:38 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:38.699+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:12.743349+0000 front 2024-03-18T23:21:12.743318+0000 (oldest deadline 2024-03-18T23:21:38.643391+0000) 2024-03-18T23:21:39.190 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:38 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:38.746+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:39.190 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:38 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:38.746+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:39.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:39.034+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:39.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:39.034+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:39.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:39.034+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:39.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:39.191+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:39.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:39.191+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:39.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:39.191+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:39.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:39.266+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:39.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:39.266+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:39.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:39.321+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:39.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:39.321+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:39.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:39 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:39.321+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:40.021 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:39.715+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:40.021 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:39.715+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:40.021 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:39.715+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:40.021 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:39 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:39.715+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:12.743349+0000 front 2024-03-18T23:21:12.743318+0000 (oldest deadline 2024-03-18T23:21:38.643391+0000) 2024-03-18T23:21:40.197 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:39.779+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:40.197 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:39.779+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:40.198 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:39 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:39.779+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:40.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:40.021+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:40.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:40.021+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:40.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:40.021+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:40.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:40.199+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:40.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:40.199+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:40.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:40.199+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:40.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:40 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:40.289+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:40.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:40 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:40.289+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:40.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:40 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:40.367+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:40.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:40 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:40.367+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:40.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:40 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:40.367+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:41.004 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:40.728+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.439960+0000 front 2024-03-18T23:20:52.439988+0000 (oldest deadline 2024-03-18T23:21:17.740161+0000) 2024-03-18T23:21:41.004 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:40.728+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:01.741487+0000 front 2024-03-18T23:21:01.741698+0000 (oldest deadline 2024-03-18T23:21:24.641467+0000) 2024-03-18T23:21:41.004 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:40.728+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.742706+0000 front 2024-03-18T23:21:08.742590+0000 (oldest deadline 2024-03-18T23:21:32.242464+0000) 2024-03-18T23:21:41.004 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:40 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6[59856]: 2024-03-18T23:21:40.728+0000 7f8338feb700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:12.743349+0000 front 2024-03-18T23:21:12.743318+0000 (oldest deadline 2024-03-18T23:21:38.643391+0000) 2024-03-18T23:21:41.204 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:40.765+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:41.204 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:40.765+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:41.204 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:40 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:40.765+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:41.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.005+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:41.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.005+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:41.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.005+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:41.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:41.205+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:41.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:41.205+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:41.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:41.205+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:41.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:41.314+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:41.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:41.314+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:41.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:41.388+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:41.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:41.388+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:41.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:41.388+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:41.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:41.722+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:41.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:41.722+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:41.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:41 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:41.722+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:42.347 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:42 smithi132 podman[92560]: 2024-03-18 23:21:42.099830439 +0000 UTC m=+5.507517578 container died 0a97582946a6dec4689d63c563466bff1aa82440c4b44486565047b539dedde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-18T23:21:42.347 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.978+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:42.347 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.978+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:42.347 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.978+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:42.347 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:41 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:41.978+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:42.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:42.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:42.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:42.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:42.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:42.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:42.331+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:42.331+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:42.331+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:42.360+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:42.360+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:42.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:42 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:42.360+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:42.613 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:42 smithi132 podman[92560]: 2024-03-18 23:21:42.438840393 +0000 UTC m=+5.846527531 container cleanup 0a97582946a6dec4689d63c563466bff1aa82440c4b44486565047b539dedde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:42.613 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:42 smithi132 bash[92560]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6 2024-03-18T23:21:42.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:42 smithi132 podman[92577]: 2024-03-18 23:21:42.615031491 +0000 UTC m=+0.511671430 container remove 0a97582946a6dec4689d63c563466bff1aa82440c4b44486565047b539dedde2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-18T23:21:43.160 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:42.725+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:43.161 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:42.725+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:43.161 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:42 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:42.725+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:43.291 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:43.049210263 +0000 UTC m=+0.109069872 container create 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T23:21:43.291 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:42.958899534 +0000 UTC m=+0.018759102 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:43.292 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:43 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:43.011+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:43.292 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:43 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:43.011+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:43.292 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:43 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:43.011+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:43.292 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:43 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:43.011+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:43.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:43.161+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:43.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:43.161+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:43.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:43.161+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:43.592 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:43.293492813 +0000 UTC m=+0.353352380 container init 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:43.592 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:43.296993243 +0000 UTC m=+0.356852812 container start 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-18T23:21:43.592 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:43.401799694 +0000 UTC m=+0.461659273 container attach 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, ceph=True) 2024-03-18T23:21:43.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:43.365+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:43.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:43.365+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:43.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:43.365+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:43.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:43.317+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:43.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:43.317+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:43.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:43 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:43.317+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:43.864 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:43 smithi132 podman[92668]: 2024-03-18 23:21:43.59413491 +0000 UTC m=+0.653994474 container died 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True) 2024-03-18T23:21:43.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:43.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:43.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:43.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:43.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:43 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:43.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:44.137 INFO:journalctl@ceph.osd.6.smithi132.stdout:Mar 18 23:21:44 smithi132 podman[92694]: 2024-03-18 23:21:44.122174141 +0000 UTC m=+0.519040760 container remove 97e447d50400c84a4c7d27c60d8df4d6d8a463f42efb130c0354f466493c8afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-18T23:21:44.137 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:44 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:44.032+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:44.138 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:44 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:44.032+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:44.138 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:44 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:44.032+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:44.138 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:44 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:44.032+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:44.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.6.service' 2024-03-18T23:21:44.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:44.184 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-18T23:21:44.184 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-18T23:21:44.184 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.7 2024-03-18T23:21:44.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:44.155+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:44.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:44.155+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:44.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:44.155+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:44.354+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:44.354+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:44.354+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:44.354+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 systemd[1]: Stopping Ceph osd.7 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:44.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.347+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:44.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.347+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:44.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.347+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:44.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.453+0000 7f9d4ed96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:44.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.453+0000 7f9d4ed96700 -1 osd.7 104 *** Got signal Terminated *** 2024-03-18T23:21:44.609 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:44.453+0000 7f9d4ed96700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:44.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:44.709+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:44.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:44.709+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:44.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:44 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:44.709+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:45.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.001+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:45.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.001+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:45.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.001+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:45.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.001+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:45.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:45.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:45.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:45.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:45.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:45.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:45.607 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:45.300+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:45.300+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:45.300+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:45.376+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:45.376+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:45.376+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:45.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:45 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:45.376+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:46.191 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:45.732+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:46.191 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:45.732+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:46.191 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:45 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:45.732+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:46.266 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.998+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:46.266 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.998+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:46.266 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.998+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:46.266 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:45 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:45.998+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:46.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:46.192+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:46.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:46.192+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:46.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:46.192+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:46.314+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:46.314+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:46.314+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:46.335+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:46.335+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:46.335+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:46.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:46 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:46.335+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:47.202 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:46.729+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:47.202 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:46.729+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:47.203 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:46.729+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:47.203 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:46 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:46.729+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:47.281 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:47.002+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:47.281 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:47.002+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:47.281 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:47.002+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:47.281 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:47 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:47.002+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:47.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:47.203+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:47.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:47.203+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:47.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:47.203+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:47.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:47.328+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:47.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:47.328+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:47.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:47.328+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:47.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:47.310+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:47.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:47.310+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:47.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:47.310+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:47.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:47 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:47.310+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:48.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:47.755+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:48.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:47.755+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:48.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:47.755+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:48.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:47 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:47.755+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:48.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:48.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:48.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:48.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:48.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:48.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:48.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:48 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:48.038+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:48.350+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:48.350+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:48.350+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:48.333+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:48.333+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:48.333+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:48.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:48.333+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:48.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:48.238+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:48.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:48.238+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:48.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:48.238+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:48.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:48.238+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:49.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:48.741+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:49.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:48.741+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:49.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:48.741+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:49.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:48 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:48.741+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:49.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:49.037+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:49.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:49.037+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:49.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:49.037+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:49.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:49 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:49.037+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:49.372+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:49.372+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:49.372+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:49.372+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:49.323+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:56.667326+0000 front 2024-03-18T23:20:56.667407+0000 (oldest deadline 2024-03-18T23:21:23.667950+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:49.323+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:09.370073+0000 front 2024-03-18T23:21:09.370086+0000 (oldest deadline 2024-03-18T23:21:35.270108+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:49.323+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.871680+0000 front 2024-03-18T23:21:16.871708+0000 (oldest deadline 2024-03-18T23:21:42.171619+0000) 2024-03-18T23:21:49.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:49 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7[85354]: 2024-03-18T23:21:49.323+0000 7f9d47382700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.072667+0000 front 2024-03-18T23:21:25.072692+0000 (oldest deadline 2024-03-18T23:21:48.572920+0000) 2024-03-18T23:21:49.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:49.278+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:49.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:49.278+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:49.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:49.278+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:49.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:49.278+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:49.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:49.723+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:49.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:49.723+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:49.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:49.723+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:49.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:49 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:49.723+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:50.244 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:49 smithi144 podman[104273]: 2024-03-18 23:21:49.993027934 +0000 UTC m=+5.615041622 container died 8143c71c6ebbd92c01fc3198873b9970c355a21535f36896efe873597f9e312b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-18T23:21:50.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.017+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:50.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.017+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:50.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.017+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:50.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.017+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:50.502 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:50 smithi144 podman[104273]: 2024-03-18 23:21:50.31997243 +0000 UTC m=+5.941986093 container cleanup 8143c71c6ebbd92c01fc3198873b9970c355a21535f36896efe873597f9e312b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T23:21:50.503 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:50 smithi144 bash[104273]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7 2024-03-18T23:21:50.503 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:50 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:50.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:50.503 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:50 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:50.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:50.503 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:50 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:50.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:50.503 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:50 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:50.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:50.701 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:50.243+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:50.701 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:50.243+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:50.701 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:50.243+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:50.701 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:50.243+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:50.857 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:50 smithi144 podman[104289]: 2024-03-18 23:21:50.504522117 +0000 UTC m=+0.506460792 container remove 8143c71c6ebbd92c01fc3198873b9970c355a21535f36896efe873597f9e312b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:21:50.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:50.702+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:50.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:50.702+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:50.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:50.702+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:50.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:50 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:50.702+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:51.214 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:50 smithi144 podman[104384]: 2024-03-18 23:21:50.83740588 +0000 UTC m=+0.017982113 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:51.214 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:50 smithi144 podman[104384]: 2024-03-18 23:21:50.938755948 +0000 UTC m=+0.119332163 container create 65ec5ddc9095c87ba28fcbfa83846c660945612a54f45743e27ce18ef153dd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T23:21:51.215 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:51 smithi144 podman[104384]: 2024-03-18 23:21:51.141453371 +0000 UTC m=+0.322029604 container init 65ec5ddc9095c87ba28fcbfa83846c660945612a54f45743e27ce18ef153dd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-18T23:21:51.215 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:51 smithi144 podman[104384]: 2024-03-18 23:21:51.144903721 +0000 UTC m=+0.325479944 container start 65ec5ddc9095c87ba28fcbfa83846c660945612a54f45743e27ce18ef153dd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:21:51.215 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:51 smithi144 podman[104384]: 2024-03-18 23:21:51.215937219 +0000 UTC m=+0.396513460 container attach 65ec5ddc9095c87ba28fcbfa83846c660945612a54f45743e27ce18ef153dd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T23:21:51.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.977+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:51.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.977+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:51.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.977+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:51.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.977+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:51.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:50 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:50.977+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:51.608 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 18 23:21:51 smithi144 podman[104384]: 2024-03-18 23:21:51.428576411 +0000 UTC m=+0.609152633 container died 65ec5ddc9095c87ba28fcbfa83846c660945612a54f45743e27ce18ef153dd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:51.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:51 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:51.419+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:51.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:51 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:51.419+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:51.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:51 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:51.419+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:51.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:51 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:51.419+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:51.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:51 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:51.419+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:51.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:51.282+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:51.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:51.282+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:51.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:51.282+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:51.692 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:51.282+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:51.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:51.692+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:51.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:51.692+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:51.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:51.692+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:51.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:51 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:51.692+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:52.008 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.7.service' 2024-03-18T23:21:52.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:52.050 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-18T23:21:52.050 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-18T23:21:52.050 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.8 2024-03-18T23:21:52.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.007+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:52.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.007+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:52.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.007+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:52.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.007+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:52.365 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.007+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:52.366 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 systemd[1]: Stopping Ceph osd.8 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:21:52.367 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:52.241+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:52.367 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:52.241+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:52.367 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:52.241+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:52.367 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:52.241+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:52.607 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:52.406+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:52.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:52.406+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:52.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:52.406+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:52.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:52.406+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:52.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:52.406+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.368+0000 7ffaee41a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.368+0000 7ffaee41a700 -1 osd.8 104 *** Got signal Terminated *** 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.368+0000 7ffaee41a700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.686+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.686+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.686+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:52.685 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:52 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:52.686+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:53.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.959+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:53.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.959+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:53.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.959+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:53.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.959+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:53.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:52 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:52.959+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:53.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:53 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:53.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:53.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:53 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:53.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:53.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:53 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:53.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:53.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:53 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:53.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:53.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:53 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:53.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:53.699 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:53.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:53.699 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:53.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:53.699 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:53.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:53.700 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:53.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:53.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:53.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:53.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:53.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:53.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:53.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:53.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:53 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:53.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:54.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:53 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:53.914+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:54.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:53 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:53.914+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:54.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:53 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:53.914+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:54.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:53 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:53.914+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:54.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:53 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:53.914+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:54.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:54 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:54.322+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:54.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:54 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:54.322+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:54.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:54 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:54.322+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:54.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:54 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:54.322+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:54.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:54 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:54.322+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:54.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:54.249+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:54.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:54.249+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:54.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:54.249+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:54.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:54.249+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:55.206 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:54.730+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:55.206 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:54.730+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:55.206 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:54.730+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:55.206 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:54.730+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:55.206 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:54.730+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:32.917334+0000 front 2024-03-18T23:21:32.917358+0000 (oldest deadline 2024-03-18T23:21:54.617466+0000) 2024-03-18T23:21:55.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:54 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:54.894+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:55.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:54 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:54.894+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:55.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:54 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:54.894+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:55.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:54 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:54.894+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:55.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:54 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:54.894+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:55.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:55.207+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:55.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:55.207+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:55.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:55.207+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:55.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:55.207+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:55.607 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:55 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:55.300+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:55.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:55 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:55.300+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:55.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:55 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:55.300+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:55.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:55 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:55.300+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:55.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:55 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:55.300+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:56.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:55.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:56.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:55.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:56.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:55.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:56.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:55.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:56.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:55 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:55.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:56.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:55.740+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:56.222 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:55.740+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:56.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:55.740+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:56.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:55.740+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:56.223 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:55 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:55.740+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:32.917334+0000 front 2024-03-18T23:21:32.917358+0000 (oldest deadline 2024-03-18T23:21:54.617466+0000) 2024-03-18T23:21:56.497 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:56.251+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:56.497 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:56.251+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:56.497 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:56.251+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:56.498 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:56.251+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:56.498 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:56.251+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:56.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:56 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:56.294+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:21:56.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:56.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.911588+0000 front 2024-03-18T23:20:59.911625+0000 (oldest deadline 2024-03-18T23:21:24.611598+0000) 2024-03-18T23:21:56.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:56.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.912658+0000 front 2024-03-18T23:21:06.912687+0000 (oldest deadline 2024-03-18T23:21:31.612870+0000) 2024-03-18T23:21:56.972 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:56.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.913982+0000 front 2024-03-18T23:21:16.914076+0000 (oldest deadline 2024-03-18T23:21:39.213950+0000) 2024-03-18T23:21:56.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:56.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:23.815720+0000 front 2024-03-18T23:21:23.815803+0000 (oldest deadline 2024-03-18T23:21:46.715536+0000) 2024-03-18T23:21:56.973 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:56 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8[85895]: 2024-03-18T23:21:56.701+0000 7ffaeaa0e700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:32.917334+0000 front 2024-03-18T23:21:32.917358+0000 (oldest deadline 2024-03-18T23:21:54.617466+0000) 2024-03-18T23:21:57.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:57.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:57.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:57.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:57.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:57.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:56 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:56.851+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:21:57.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:57 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:57.221+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:57.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:57 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:57.221+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:57.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:57 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:57.221+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:57.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:57 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:57.221+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:57.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:57 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:57.221+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:57.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:57 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:57.312+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:21:58.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:58.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:58.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:58.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:58.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:58.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:57 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:57.804+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:21:58.169 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:57 smithi196 podman[104937]: 2024-03-18 23:21:57.854528397 +0000 UTC m=+5.605874217 container died 05fa8d9ec3cf7bc7a68df704bebf480838d0fd291ae8995dfb129854df0f6f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:58 smithi196 podman[104937]: 2024-03-18 23:21:58.26164255 +0000 UTC m=+6.012988368 container cleanup 05fa8d9ec3cf7bc7a68df704bebf480838d0fd291ae8995dfb129854df0f6f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:58 smithi196 bash[104937]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:58 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:58.182+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:58 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:58.182+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:58 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:58.182+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:58 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:58.182+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:58.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:58 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:58.182+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:21:58.770 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:58 smithi196 podman[104968]: 2024-03-18 23:21:58.479437131 +0000 UTC m=+0.617048859 container remove 05fa8d9ec3cf7bc7a68df704bebf480838d0fd291ae8995dfb129854df0f6f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T23:21:58.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:58.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:58.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:58.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:58.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:58.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:58 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:58.358+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:21:58.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:58 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:58.756+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:21:59.081 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:58 smithi196 podman[105058]: 2024-03-18 23:21:58.772035402 +0000 UTC m=+0.112775261 container create 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-18T23:21:59.081 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:58 smithi196 podman[105058]: 2024-03-18 23:21:58.676587142 +0000 UTC m=+0.017326974 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:21:59.081 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:59 smithi196 podman[105058]: 2024-03-18 23:21:59.015769551 +0000 UTC m=+0.356509384 container init 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-18T23:21:59.081 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:59 smithi196 podman[105058]: 2024-03-18 23:21:59.018833267 +0000 UTC m=+0.359573076 container start 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T23:21:59.081 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:59 smithi196 podman[105058]: 2024-03-18 23:21:59.082311832 +0000 UTC m=+0.423051662 container attach 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T23:21:59.472 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:59 smithi196 podman[105058]: 2024-03-18 23:21:59.30668052 +0000 UTC m=+0.647420337 container died 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T23:21:59.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:59 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:59.132+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:21:59.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:59 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:59.132+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:21:59.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:59 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:59.132+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:21:59.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:59 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:59.132+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:21:59.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:21:59 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:21:59.132+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:21:59.826 INFO:journalctl@ceph.osd.8.smithi196.stdout:Mar 18 23:21:59 smithi196 podman[105085]: 2024-03-18 23:21:59.81117853 +0000 UTC m=+0.499012244 container remove 7cc22f9325aaf7d6077a76fb7da5ec856d5c4ed3da816ae9a5b8fb41135dc9f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-18T23:21:59.832 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.8.service' 2024-03-18T23:21:59.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:21:59.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:21:59.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:21:59.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:21:59.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:21:59.859 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:21:59 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:21:59.387+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:21:59.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:21:59.875 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-18T23:21:59.875 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-18T23:21:59.876 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.9 2024-03-18T23:22:00.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:00.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:00.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:00.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:00.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:00.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:21:59.718+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:00.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:21:59 smithi132 systemd[1]: Stopping Ceph osd.9 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:00.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:00.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:00.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:00.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:00.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:00.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:00.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:00.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:00.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:00.158+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:00.614 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.182+0000 7fc29b53d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:00.614 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.182+0000 7fc29b53d700 -1 osd.9 104 *** Got signal Terminated *** 2024-03-18T23:22:00.614 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.182+0000 7fc29b53d700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:22:00.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:00.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:00.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:00.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:00.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:00.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:00 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:00.379+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:01.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:01.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:01.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:01.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:01.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:01.115 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:00 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:00.767+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:01.222 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:01 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:01.142+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:01.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:01 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:01.142+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:01.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:01 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:01.142+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:01.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:01 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:01.142+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:01.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:01 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:01.142+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:01.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:01 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:01.400+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:02.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:01 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:01.727+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:02.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:02 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:02.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:02.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:02 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:02.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:02.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:02 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:02.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:02.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:02 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:02.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:02.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:02 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:02.189+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:02.692 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:02.693 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:02.693 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:02.693 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:02.693 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:02.693 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:02 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:02.402+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:03.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:02 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:02.768+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:03.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:03 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:03.409+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:03.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:03 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:03.226+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:03.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:03 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:03.226+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:03.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:03 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:03.226+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:03.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:03 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:03.226+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:03.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:03 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:03.226+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:04.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:03 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:03.786+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:04.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:04.225+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:04.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:04.225+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:04.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:04.225+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:04.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:04.225+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:04.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:04.225+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:04.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:04 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:04.420+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:52.287177+0000 front 2024-03-18T23:20:52.287291+0000 (oldest deadline 2024-03-18T23:21:16.987368+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:59.888754+0000 front 2024-03-18T23:20:59.888567+0000 (oldest deadline 2024-03-18T23:21:23.388650+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:06.290053+0000 front 2024-03-18T23:21:06.290154+0000 (oldest deadline 2024-03-18T23:21:32.189787+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:16.191099+0000 front 2024-03-18T23:21:16.191123+0000 (oldest deadline 2024-03-18T23:21:41.491212+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.492933+0000 front 2024-03-18T23:21:25.493067+0000 (oldest deadline 2024-03-18T23:21:50.792783+0000) 2024-03-18T23:22:05.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:04 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9[66882]: 2024-03-18T23:22:04.819+0000 7fc293b29700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.793296+0000 front 2024-03-18T23:21:30.793373+0000 (oldest deadline 2024-03-18T23:21:56.093272+0000) 2024-03-18T23:22:05.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:05.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:05.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:05.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:05.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:05.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:05 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:05.231+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:05.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:05 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:05.426+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:05.864 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:05 smithi132 podman[92788]: 2024-03-18 23:22:05.612895981 +0000 UTC m=+5.534601703 container died 57e5a58240515067dfcdfa72f7f2c5a05f03f90039695eb3fc8886d02fc70de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:22:06.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:05 smithi132 podman[92788]: 2024-03-18 23:22:05.948882756 +0000 UTC m=+5.870588478 container cleanup 57e5a58240515067dfcdfa72f7f2c5a05f03f90039695eb3fc8886d02fc70de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:22:06.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:05 smithi132 bash[92788]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9 2024-03-18T23:22:06.364 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92806]: 2024-03-18 23:22:06.133267632 +0000 UTC m=+0.514543341 container remove 57e5a58240515067dfcdfa72f7f2c5a05f03f90039695eb3fc8886d02fc70de9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T23:22:06.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:06.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:06.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:06.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:06.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:06.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:06 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:06.210+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:06.835 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92896]: 2024-03-18 23:22:06.460270169 +0000 UTC m=+0.018860861 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:22:06.835 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92896]: 2024-03-18 23:22:06.567756894 +0000 UTC m=+0.126347578 container create 2609ad9d87cb4c74a4e246f1fbd7b38f1e931d2d16eace4cefe3a215f00c41a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-03-18T23:22:06.835 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92896]: 2024-03-18 23:22:06.76177866 +0000 UTC m=+0.320369355 container init 2609ad9d87cb4c74a4e246f1fbd7b38f1e931d2d16eace4cefe3a215f00c41a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T23:22:06.835 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92896]: 2024-03-18 23:22:06.764964326 +0000 UTC m=+0.323555008 container start 2609ad9d87cb4c74a4e246f1fbd7b38f1e931d2d16eace4cefe3a215f00c41a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-18T23:22:06.835 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:06 smithi132 podman[92896]: 2024-03-18 23:22:06.836777786 +0000 UTC m=+0.395368485 container attach 2609ad9d87cb4c74a4e246f1fbd7b38f1e931d2d16eace4cefe3a215f00c41a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T23:22:06.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:06.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:06.857 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:06.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:06.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:06.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:06.858 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:06 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:06.384+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:07.114 INFO:journalctl@ceph.osd.9.smithi132.stdout:Mar 18 23:22:07 smithi132 podman[92896]: 2024-03-18 23:22:07.055430039 +0000 UTC m=+0.614020710 container died 2609ad9d87cb4c74a4e246f1fbd7b38f1e931d2d16eace4cefe3a215f00c41a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:22:07.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.9.service' 2024-03-18T23:22:07.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:07.679 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-18T23:22:07.679 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-18T23:22:07.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.10 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:07.709 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:07.710 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.365+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:07.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:07.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:07.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:07.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:07.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:07.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:07 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:07.239+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:07.994 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 systemd[1]: Stopping Ceph osd.10 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.994+0000 7faa70147700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.994+0000 7faa70147700 -1 osd.10 104 *** Got signal Terminated *** 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:07 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:07.994+0000 7faa70147700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:08.322 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:08.323 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:08.323 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:08.323 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:08 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:08.323+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:08.472 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:08.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:08.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:08.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:08.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:08.473 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:08.215+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:09.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:09 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:09.281+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:09.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:09.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:09.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:09.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:09.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:09.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:09 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:09.244+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:10.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:10.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:10.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:10.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:10.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:10.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:10.609 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:10 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:10.263+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:10.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:10.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:10.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:10.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:10.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:10.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:10 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:10.270+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:11.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:11 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:11.232+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:11.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:11.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:11.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:11.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:11.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:11.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:11 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:11.299+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:50.194139+0000 front 2024-03-18T23:20:50.194241+0000 (oldest deadline 2024-03-18T23:21:16.094228+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:21:00.696047+0000 front 2024-03-18T23:21:00.696069+0000 (oldest deadline 2024-03-18T23:21:24.196175+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:10.097214+0000 front 2024-03-18T23:21:10.097335+0000 (oldest deadline 2024-03-18T23:21:31.797245+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6814 osd.3 since back 2024-03-18T23:21:15.898273+0000 front 2024-03-18T23:21:15.898245+0000 (oldest deadline 2024-03-18T23:21:44.098782+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.199886+0000 front 2024-03-18T23:21:25.199920+0000 (oldest deadline 2024-03-18T23:21:50.499945+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:30.500506+0000 front 2024-03-18T23:21:30.500410+0000 (oldest deadline 2024-03-18T23:21:55.800258+0000) 2024-03-18T23:22:12.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:12 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10[92432]: 2024-03-18T23:22:12.276+0000 7faa68733700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.801668+0000 front 2024-03-18T23:21:39.801603+0000 (oldest deadline 2024-03-18T23:22:03.301409+0000) 2024-03-18T23:22:12.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:12.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:12 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:12.340+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:13.607 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:13 smithi144 podman[104499]: 2024-03-18 23:22:13.356165935 +0000 UTC m=+5.481532165 container died 266983be1ae610c35dd08665d6523691f7bc5432714e715523c783819ab96300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T23:22:13.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:13.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:13 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:13.335+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:13.970 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:13 smithi144 podman[104499]: 2024-03-18 23:22:13.703519871 +0000 UTC m=+5.828886100 container cleanup 266983be1ae610c35dd08665d6523691f7bc5432714e715523c783819ab96300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:22:13.970 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:13 smithi144 bash[104499]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10 2024-03-18T23:22:14.318 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:13 smithi144 podman[104517]: 2024-03-18 23:22:13.971224601 +0000 UTC m=+0.616082479 container remove 266983be1ae610c35dd08665d6523691f7bc5432714e715523c783819ab96300 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T23:22:14.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.222802185 +0000 UTC m=+0.017146301 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:22:14.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.330177981 +0000 UTC m=+0.124522098 container create b62d5c1ab9e3f6ad51ca916c6209d34b5586c650f3e54bdd50485974cbab881a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T23:22:14.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.565842721 +0000 UTC m=+0.360186838 container init b62d5c1ab9e3f6ad51ca916c6209d34b5586c650f3e54bdd50485974cbab881a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-18T23:22:14.608 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.568954558 +0000 UTC m=+0.363298683 container start b62d5c1ab9e3f6ad51ca916c6209d34b5586c650f3e54bdd50485974cbab881a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-18T23:22:14.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:14.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:14 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:14.355+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:15.108 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.637966948 +0000 UTC m=+0.432311071 container attach b62d5c1ab9e3f6ad51ca916c6209d34b5586c650f3e54bdd50485974cbab881a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-18T23:22:15.108 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 18 23:22:14 smithi144 podman[104610]: 2024-03-18 23:22:14.85409205 +0000 UTC m=+0.648436166 container died b62d5c1ab9e3f6ad51ca916c6209d34b5586c650f3e54bdd50485974cbab881a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T23:22:15.415 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.10.service' 2024-03-18T23:22:15.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:15.496 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-18T23:22:15.496 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-18T23:22:15.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.11 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.356+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:15.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 systemd[1]: Stopping Ceph osd.11 for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:16.222 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.769+0000 7f8d2ed9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:16.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.769+0000 7f8d2ed9e700 -1 osd.11 104 *** Got signal Terminated *** 2024-03-18T23:22:16.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:15 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:15.769+0000 7f8d2ed9e700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T23:22:16.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:16.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:16 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:16.362+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:17.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:17.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:17 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:17.399+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:18.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:18.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:18 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:18.388+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-03-18T23:21:52.225053+0000 front 2024-03-18T23:21:52.225018+0000 (oldest deadline 2024-03-18T23:22:18.125225+0000) 2024-03-18T23:22:19.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:19.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:19 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:19.416+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-03-18T23:21:52.225053+0000 front 2024-03-18T23:21:52.225018+0000 (oldest deadline 2024-03-18T23:22:18.125225+0000) 2024-03-18T23:22:20.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6806 osd.0 since back 2024-03-18T23:20:54.016269+0000 front 2024-03-18T23:20:54.016107+0000 (oldest deadline 2024-03-18T23:21:18.116153+0000) 2024-03-18T23:22:20.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-18T23:20:58.617171+0000 front 2024-03-18T23:20:58.617023+0000 (oldest deadline 2024-03-18T23:21:24.517195+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-03-18T23:21:08.018305+0000 front 2024-03-18T23:21:08.018059+0000 (oldest deadline 2024-03-18T23:21:31.518173+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-18T23:21:25.620270+0000 front 2024-03-18T23:21:25.620196+0000 (oldest deadline 2024-03-18T23:21:47.320066+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-03-18T23:21:33.522497+0000 front 2024-03-18T23:21:33.522573+0000 (oldest deadline 2024-03-18T23:21:55.822665+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.132:6822 osd.6 since back 2024-03-18T23:21:39.923876+0000 front 2024-03-18T23:21:39.923891+0000 (oldest deadline 2024-03-18T23:22:04.623717+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-18T23:21:46.324696+0000 front 2024-03-18T23:21:46.324836+0000 (oldest deadline 2024-03-18T23:22:12.224497+0000) 2024-03-18T23:22:20.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:20 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11[92670]: 2024-03-18T23:22:20.434+0000 7f8d2738a700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-03-18T23:21:52.225053+0000 front 2024-03-18T23:21:52.225018+0000 (oldest deadline 2024-03-18T23:22:18.125225+0000) 2024-03-18T23:22:21.410 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 podman[105163]: 2024-03-18 23:22:21.108677592 +0000 UTC m=+5.423824719 container died cf88be764c4b4988c38491541ef2e8eafba5103dad8d3e3b01f37362370fd7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T23:22:21.410 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 podman[105163]: 2024-03-18 23:22:21.411459731 +0000 UTC m=+5.726606863 container cleanup cf88be764c4b4988c38491541ef2e8eafba5103dad8d3e3b01f37362370fd7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T23:22:21.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 bash[105163]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11 2024-03-18T23:22:21.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 podman[105181]: 2024-03-18 23:22:21.646264467 +0000 UTC m=+0.533418209 container remove cf88be764c4b4988c38491541ef2e8eafba5103dad8d3e3b01f37362370fd7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-18T23:22:22.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 podman[105270]: 2024-03-18 23:22:21.964913185 +0000 UTC m=+0.105153430 container create 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:22:22.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:21 smithi196 podman[105270]: 2024-03-18 23:22:21.877706323 +0000 UTC m=+0.017946568 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-18T23:22:22.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 podman[105270]: 2024-03-18 23:22:22.183581153 +0000 UTC m=+0.323821395 container init 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-18T23:22:22.223 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 podman[105270]: 2024-03-18 23:22:22.186677114 +0000 UTC m=+0.326917340 container start 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T23:22:22.722 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 podman[105270]: 2024-03-18 23:22:22.258639975 +0000 UTC m=+0.398880210 container attach 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:22:22.723 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 podman[105270]: 2024-03-18 23:22:22.476603983 +0000 UTC m=+0.616844201 container died 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-18T23:22:22.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.11.service' 2024-03-18T23:22:23.021 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 podman[105295]: 2024-03-18 23:22:22.978479092 +0000 UTC m=+0.496482536 container remove 8536a6e2a58c7acb92e440efc8deaa3d1f8be863dcbdcab32514c2e996a6274f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-18T23:22:23.021 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.11.service: Deactivated successfully. 2024-03-18T23:22:23.021 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 systemd[1]: Stopped Ceph osd.11 for 0a41095a-e57b-11ee-95c9-87774f69a715. 2024-03-18T23:22:23.021 INFO:journalctl@ceph.osd.11.smithi196.stdout:Mar 18 23:22:22 smithi196 systemd[1]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715@osd.11.service: Consumed 6.605s CPU time. 2024-03-18T23:22:23.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:23.036 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-18T23:22:23.036 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-18T23:22:23.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.a 2024-03-18T23:22:23.349 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:23 smithi132 systemd[1]: Stopping Ceph mds.a for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:23.614 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:23 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a[73621]: 2024-03-18T23:22:23.350+0000 7f526dc8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:23.614 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:23 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a[73621]: 2024-03-18T23:22:23.350+0000 7f526dc8c700 -1 mds.a *** got signal Terminated *** 2024-03-18T23:22:26.614 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:26 smithi132 podman[93017]: 2024-03-18 23:22:26.364383641 +0000 UTC m=+3.136044869 container died e1cfe9ae8e6f6a88c0f983fea7590d43be861c7e0274f80825623fa1e1a8a3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-03-18T23:22:26.999 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:26 smithi132 podman[93017]: 2024-03-18 23:22:26.742156348 +0000 UTC m=+3.513817576 container cleanup e1cfe9ae8e6f6a88c0f983fea7590d43be861c7e0274f80825623fa1e1a8a3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-18T23:22:26.999 INFO:journalctl@ceph.mds.a.smithi132.stdout:Mar 18 23:22:26 smithi132 bash[93017]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-a 2024-03-18T23:22:27.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.a.service' 2024-03-18T23:22:27.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:27.156 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-18T23:22:27.156 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-18T23:22:27.156 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.d 2024-03-18T23:22:27.364 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:27 smithi132 systemd[1]: Stopping Ceph mds.d for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:27.864 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d[72666]: 2024-03-18T23:22:27.478+0000 7febd3d44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:27.864 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:27 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d[72666]: 2024-03-18T23:22:27.478+0000 7febd3d44700 -1 mds.d *** got signal Terminated *** 2024-03-18T23:22:31.528 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:31 smithi132 podman[93139]: 2024-03-18 23:22:31.271729721 +0000 UTC m=+3.898149411 container died 30ccf165fb8dbb062e4c74ddf56f6a5866696681d7f594655573603799afdd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-18T23:22:31.864 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:31 smithi132 podman[93139]: 2024-03-18 23:22:31.645772293 +0000 UTC m=+4.272191982 container cleanup 30ccf165fb8dbb062e4c74ddf56f6a5866696681d7f594655573603799afdd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:22:31.864 INFO:journalctl@ceph.mds.d.smithi132.stdout:Mar 18 23:22:31 smithi132 bash[93139]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-d 2024-03-18T23:22:32.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.d.service' 2024-03-18T23:22:32.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:32.045 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-18T23:22:32.045 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-18T23:22:32.045 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.g 2024-03-18T23:22:32.614 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:32 smithi132 systemd[1]: Stopping Ceph mds.g for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:32.614 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g[73260]: 2024-03-18T23:22:32.382+0000 7f9d449b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:32.614 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:32 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g[73260]: 2024-03-18T23:22:32.382+0000 7f9d449b6700 -1 mds.g *** got signal Terminated *** 2024-03-18T23:22:35.898 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:35 smithi132 podman[93261]: 2024-03-18 23:22:35.620265821 +0000 UTC m=+3.345823409 container died e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T23:22:36.249 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:35 smithi132 podman[93261]: 2024-03-18 23:22:35.991236078 +0000 UTC m=+3.716793669 container cleanup e81fcc0c83d3504abeaed0f2ab4488c58a11b9d636e26574c8af18483f53599a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T23:22:36.249 INFO:journalctl@ceph.mds.g.smithi132.stdout:Mar 18 23:22:35 smithi132 bash[93261]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-g 2024-03-18T23:22:36.410 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.g.service' 2024-03-18T23:22:36.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:36.444 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-18T23:22:36.444 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-18T23:22:36.444 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.j 2024-03-18T23:22:36.826 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:36 smithi132 systemd[1]: Stopping Ceph mds.j for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:37.114 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j[72276]: 2024-03-18T23:22:36.827+0000 7f671162a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:37.114 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:36 smithi132 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j[72276]: 2024-03-18T23:22:36.827+0000 7f671162a700 -1 mds.j *** got signal Terminated *** 2024-03-18T23:22:40.786 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:40 smithi132 podman[93382]: 2024-03-18 23:22:40.508365671 +0000 UTC m=+3.839016916 container died c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-18T23:22:41.114 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:40 smithi132 podman[93382]: 2024-03-18 23:22:40.853311935 +0000 UTC m=+4.183963222 container cleanup c4ee62bb361c7b7a2bcf025b522d6c9c3eb98f219d701e37cf2bad13be58b5cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-18T23:22:41.114 INFO:journalctl@ceph.mds.j.smithi132.stdout:Mar 18 23:22:40 smithi132 bash[93382]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-j 2024-03-18T23:22:41.324 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.j.service' 2024-03-18T23:22:41.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:41.358 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-18T23:22:41.358 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-18T23:22:41.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.b 2024-03-18T23:22:41.651 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:41 smithi144 systemd[1]: Stopping Ceph mds.b for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:41.914 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: 2024-03-18T23:22:41.651+0000 7f1f043af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:41.914 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:41 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b[98124]: 2024-03-18T23:22:41.651+0000 7f1f043af700 -1 mds.b *** got signal Terminated *** 2024-03-18T23:22:43.668 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:43 smithi144 podman[104714]: 2024-03-18 23:22:43.405250115 +0000 UTC m=+1.855726816 container died 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-18T23:22:44.034 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:43 smithi144 podman[104714]: 2024-03-18 23:22:43.76797667 +0000 UTC m=+2.218453371 container cleanup 4215ab18d2adb5a0c48efd49f1438464405a104b00c6cc0d1e7f03ea38d58621 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T23:22:44.034 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 18 23:22:43 smithi144 bash[104714]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-b 2024-03-18T23:22:44.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.b.service' 2024-03-18T23:22:44.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:44.180 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-18T23:22:44.181 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-18T23:22:44.181 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.e 2024-03-18T23:22:44.357 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:44 smithi144 systemd[1]: Stopping Ceph mds.e for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:44.858 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: 2024-03-18T23:22:44.504+0000 7f49c055d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:44.858 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:44 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e[97500]: 2024-03-18T23:22:44.504+0000 7f49c055d700 -1 mds.e *** got signal Terminated *** 2024-03-18T23:22:47.653 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:47 smithi144 podman[104848]: 2024-03-18 23:22:47.394051547 +0000 UTC m=+2.989952641 container died 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:22:47.945 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:47 smithi144 podman[104848]: 2024-03-18 23:22:47.729859923 +0000 UTC m=+3.325761000 container cleanup 460f266c3d516408d75f8267e8793fd1361f0e08a705ad6f5b7f0396445388a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-18T23:22:47.946 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 18 23:22:47 smithi144 bash[104848]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-e 2024-03-18T23:22:48.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.e.service' 2024-03-18T23:22:48.178 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:48.178 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-18T23:22:48.178 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-18T23:22:48.179 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.h 2024-03-18T23:22:48.607 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:48 smithi144 systemd[1]: Stopping Ceph mds.h for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:48.608 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: 2024-03-18T23:22:48.482+0000 7f8697aaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:48.608 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:48 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h[97813]: 2024-03-18T23:22:48.482+0000 7f8697aaf700 -1 mds.h *** got signal Terminated *** 2024-03-18T23:22:51.691 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:51 smithi144 podman[104970]: 2024-03-18 23:22:51.400285507 +0000 UTC m=+3.007310416 container died 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-18T23:22:52.033 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:51 smithi144 podman[104970]: 2024-03-18 23:22:51.791794031 +0000 UTC m=+3.398818988 container cleanup 19b963446709763b45a33135797854b2c96b8b20d2b3484a949debf96a602a1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-18T23:22:52.033 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 18 23:22:51 smithi144 bash[104970]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-h 2024-03-18T23:22:52.153 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.h.service' 2024-03-18T23:22:52.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:52.188 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-18T23:22:52.188 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-18T23:22:52.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.k 2024-03-18T23:22:52.337 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:52 smithi144 systemd[1]: Stopping Ceph mds.k for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:52.607 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k[97175]: 2024-03-18T23:22:52.535+0000 7f83489d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:52.608 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:52 smithi144 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k[97175]: 2024-03-18T23:22:52.535+0000 7f83489d2700 -1 mds.k *** got signal Terminated *** 2024-03-18T23:22:54.035 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:53 smithi144 podman[105092]: 2024-03-18 23:22:53.775788772 +0000 UTC m=+1.362448572 container died 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T23:22:54.305 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:54 smithi144 podman[105092]: 2024-03-18 23:22:54.128500195 +0000 UTC m=+1.715159994 container cleanup 7cc6ceffd3accd414aa243e9a45c4c9877c2db75c05d86095fb62dc4201301fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:22:54.305 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 18 23:22:54 smithi144 bash[105092]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-k 2024-03-18T23:22:54.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.k.service' 2024-03-18T23:22:54.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:54.562 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-18T23:22:54.562 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-18T23:22:54.562 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.c 2024-03-18T23:22:54.918 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:54 smithi196 systemd[1]: Stopping Ceph mds.c for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:22:55.176 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: 2024-03-18T23:22:54.918+0000 7f0d34938700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:22:55.176 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:54 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c[97716]: 2024-03-18T23:22:54.918+0000 7f0d34938700 -1 mds.c *** got signal Terminated *** 2024-03-18T23:22:59.222 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:58 smithi196 podman[105374]: 2024-03-18 23:22:58.933432017 +0000 UTC m=+4.178947188 container died 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:22:59.843 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:59 smithi196 podman[105374]: 2024-03-18 23:22:59.619256016 +0000 UTC m=+4.864771184 container cleanup 0131005cfb2fd03a4436ceabf9901dec86f91b2192487666a1a7d6586f59f7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-18T23:22:59.843 INFO:journalctl@ceph.mds.c.smithi196.stdout:Mar 18 23:22:59 smithi196 bash[105374]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-c 2024-03-18T23:22:59.957 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.c.service' 2024-03-18T23:22:59.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:22:59.996 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-18T23:22:59.996 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-18T23:22:59.996 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.f 2024-03-18T23:23:00.222 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:00 smithi196 systemd[1]: Stopping Ceph mds.f for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:23:00.722 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: 2024-03-18T23:23:00.305+0000 7f79d5289700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:23:00.722 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:00 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f[97405]: 2024-03-18T23:23:00.305+0000 7f79d5289700 -1 mds.f *** got signal Terminated *** 2024-03-18T23:23:03.722 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:03 smithi196 podman[105506]: 2024-03-18 23:23:03.400770737 +0000 UTC m=+3.187953544 container died 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:23:04.038 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:03 smithi196 podman[105506]: 2024-03-18 23:23:03.813770982 +0000 UTC m=+3.600953813 container cleanup 8977edd44b39500126b77e7f9ca8aa02b5d588433325a7d8c7337bd7e607b84a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:23:04.038 INFO:journalctl@ceph.mds.f.smithi196.stdout:Mar 18 23:23:03 smithi196 bash[105506]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-f 2024-03-18T23:23:04.163 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.f.service' 2024-03-18T23:23:04.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:23:04.199 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-18T23:23:04.199 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-18T23:23:04.199 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.i 2024-03-18T23:23:04.472 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:04 smithi196 systemd[1]: Stopping Ceph mds.i for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:23:04.972 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: 2024-03-18T23:23:04.501+0000 7f16385bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:23:04.972 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:04 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i[98022]: 2024-03-18T23:23:04.501+0000 7f16385bc700 -1 mds.i *** got signal Terminated *** 2024-03-18T23:23:07.722 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:07 smithi196 podman[105626]: 2024-03-18 23:23:07.410377817 +0000 UTC m=+3.000842359 container died 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:23:08.008 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:07 smithi196 podman[105626]: 2024-03-18 23:23:07.808943681 +0000 UTC m=+3.399408213 container cleanup 963dfdec843cf9dfd9bf32eed99b8051f1c406a74e613f9cc4d50d552955de02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-18T23:23:08.008 INFO:journalctl@ceph.mds.i.smithi196.stdout:Mar 18 23:23:07 smithi196 bash[105626]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-i 2024-03-18T23:23:08.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.i.service' 2024-03-18T23:23:08.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:23:08.177 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-18T23:23:08.177 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-18T23:23:08.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.l 2024-03-18T23:23:08.722 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:08 smithi196 systemd[1]: Stopping Ceph mds.l for 0a41095a-e57b-11ee-95c9-87774f69a715... 2024-03-18T23:23:08.722 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l[97080]: 2024-03-18T23:23:08.494+0000 7ffa574af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T23:23:08.722 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:08 smithi196 ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l[97080]: 2024-03-18T23:23:08.494+0000 7ffa574af700 -1 mds.l *** got signal Terminated *** 2024-03-18T23:23:11.472 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:11 smithi196 podman[105748]: 2024-03-18 23:23:11.179657763 +0000 UTC m=+2.784922018 container died 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-18T23:23:11.885 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:11 smithi196 podman[105748]: 2024-03-18 23:23:11.487030057 +0000 UTC m=+3.092294296 container cleanup 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T23:23:11.885 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:11 smithi196 bash[105748]: ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l 2024-03-18T23:23:11.885 INFO:journalctl@ceph.mds.l.smithi196.stdout:Mar 18 23:23:11 smithi196 podman[105763]: 2024-03-18 23:23:11.662806323 +0000 UTC m=+0.485685271 container remove 12f87cf7218ffa84d75347307ef6a9f95640102f777ff78cdbee76a637985406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-0a41095a-e57b-11ee-95c9-87774f69a715-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T23:23:11.909 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0a41095a-e57b-11ee-95c9-87774f69a715@mds.l.service' 2024-03-18T23:23:11.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T23:23:11.943 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-18T23:23:11.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:23:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:14.626 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:23:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:17.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T23:23:17.486 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:19.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:23:19.895 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:23:19.930 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T23:23:19.959 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-18T23:23:19.961 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi132/crash 2024-03-18T23:23:19.962 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:23:20.026 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:23:20.026 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:23:20.028 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi144/crash 2024-03-18T23:23:20.029 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:23:20.070 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:23:20.071 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:23:20.072 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi196/crash 2024-03-18T23:23:20.073 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash -- . 2024-03-18T23:23:20.113 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T23:23:20.113 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-03-18T23:23:20.115 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-18T23:23:20.115 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-18T23:23:20.153 INFO:teuthology.orchestra.run.smithi132.stdout:2024-03-18T23:01:18.036932+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:23:20.153 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-18T23:23:20.154 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[SEC\]' /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-18T23:23:20.228 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[ERR\]' /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-18T23:23:20.300 DEBUG:teuthology.orchestra.run.smithi132:> sudo egrep '\[WRN\]' /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-18T23:23:20.373 INFO:teuthology.orchestra.run.smithi132.stdout:2024-03-18T23:01:18.036932+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T23:23:20.373 INFO:tasks.cephadm:Compressing logs... 2024-03-18T23:23:20.374 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:23:20.416 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:23:20.418 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T23:23:20.440 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T23:23:20.441 INFO:teuthology.orchestra.run.smithi132.stderr:: No such file or directory 2024-03-18T23:23:20.441 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-18T23:23:20.441 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-18T23:23:20.441 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:23:20.443 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.6.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-18T23:23:20.443 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-18T23:23:20.443 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-18T23:23:20.443 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-18T23:23:20.443 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-18T23:23:20.444 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T23:23:20.444 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-03-18T23:23:20.444 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-18T23:23:20.447 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:23:20.447 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:23:20.447 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-18T23:23:20.447 INFO:teuthology.orchestra.run.smithi132.stderr: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-18T23:23:20.450 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:23:20.450 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-18T23:23:20.450 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.l.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-18T23:23:20.450 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.11.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-18T23:23:20.451 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-18T23:23:20.454 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.2.log: 87.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:23:20.455 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:23:20.457 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:23:20.457 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:23:20.458 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:23:20.458 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log: 84.1% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T23:23:20.458 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-18T23:23:20.458 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-18T23:23:20.458 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.9.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log: 87.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:23:20.460 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-18T23:23:20.460 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.b.log: 92.1%gzip -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr: -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-18T23:23:20.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-18T23:23:20.462 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.7.log: 90.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T23:23:20.462 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-18T23:23:20.462 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-18T23:23:20.462 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T23:23:20.466 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-18T23:23:20.466 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:23:20.467 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.y.log: 91.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-18T23:23:20.467 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log 2024-03-18T23:23:20.467 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T23:23:20.468 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T23:23:20.470 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-18T23:23:20.471 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log: 87.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T23:23:20.471 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-18T23:23:20.471 INFO:teuthology.orchestra.run.smithi132.stderr: 94.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-18T23:23:20.472 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:23:20.484 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T23:23:20.511 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.b.log: 93.2% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-18T23:23:20.517 INFO:teuthology.orchestra.run.smithi144.stderr: 94.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-18T23:23:20.552 INFO:teuthology.orchestra.run.smithi196.stderr: 92.7% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-18T23:23:20.603 INFO:teuthology.orchestra.run.smithi196.stderr: 95.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-18T23:23:20.767 INFO:teuthology.orchestra.run.smithi132.stderr: 89.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-18T23:23:20.879 INFO:teuthology.orchestra.run.smithi132.stderr: 92.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-18T23:23:20.882 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-18T23:23:20.895 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-18T23:23:20.908 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-18T23:23:20.948 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-18T23:23:20.978 INFO:teuthology.orchestra.run.smithi196.stderr: 93.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-18T23:23:21.009 INFO:teuthology.orchestra.run.smithi132.stderr: 93.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-18T23:23:21.030 INFO:teuthology.orchestra.run.smithi144.stderr: 92.3% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-18T23:23:21.077 INFO:teuthology.orchestra.run.smithi144.stderr: 93.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-18T23:23:21.089 INFO:teuthology.orchestra.run.smithi132.stderr: 92.7% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-18T23:23:21.140 INFO:teuthology.orchestra.run.smithi132.stderr: 92.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-18T23:23:21.158 INFO:teuthology.orchestra.run.smithi144.stderr: 92.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-18T23:23:21.238 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-18T23:23:21.307 INFO:teuthology.orchestra.run.smithi132.stderr: 90.5% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-18T23:23:21.319 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-03-18T23:23:21.319 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m0.882s 2024-03-18T23:23:21.319 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m3.463s 2024-03-18T23:23:21.319 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m0.143s 2024-03-18T23:23:21.363 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-18T23:23:21.367 INFO:teuthology.orchestra.run.smithi144.stderr: 93.0% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-18T23:23:21.369 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-18T23:23:21.369 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m0.937s 2024-03-18T23:23:21.369 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m3.740s 2024-03-18T23:23:21.369 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.151s 2024-03-18T23:23:21.792 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/0a41095a-e57b-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-18T23:23:21.794 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-03-18T23:23:21.794 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m1.362s 2024-03-18T23:23:21.794 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m4.645s 2024-03-18T23:23:21.794 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.179s 2024-03-18T23:23:21.794 INFO:tasks.cephadm:Archiving logs... 2024-03-18T23:23:21.794 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi132/log 2024-03-18T23:23:21.795 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:23:22.329 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi144/log 2024-03-18T23:23:22.330 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:23:22.841 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi196/log 2024-03-18T23:23:22.843 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T23:23:23.457 INFO:tasks.cephadm:Removing cluster... 2024-03-18T23:23:23.457 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force 2024-03-18T23:23:23.650 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:23.918 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force 2024-03-18T23:23:24.110 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:24.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0a41095a-e57b-11ee-95c9-87774f69a715 --force 2024-03-18T23:23:24.557 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 0a41095a-e57b-11ee-95c9-87774f69a715 2024-03-18T23:23:24.849 INFO:tasks.cephadm:Removing cephadm ... 2024-03-18T23:23:24.849 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:23:24.867 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:23:24.884 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T23:23:24.899 INFO:tasks.cephadm:Teardown complete 2024-03-18T23:23:24.899 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T23:23:24.911 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T23:23:24.912 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:23:24.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:23:24.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T23:23:25.045 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T23:23:25.058 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T23:23:25.059 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:23:25.061 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:23:25.064 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T23:23:25.074 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:23:25.076 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:23:25.077 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1474us[+1487us] +/- 121ms 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -132us[ -119us] +/- 54ms 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -465us[ -465us] +/- 62ms 2024-03-18T23:23:25.079 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:23:25.080 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:23:25.080 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-18T23:23:25.080 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +1428us[+1407us] +/- 121ms 2024-03-18T23:23:25.081 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -175us[ -196us] +/- 54ms 2024-03-18T23:23:25.081 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -484us[ -484us] +/- 62ms 2024-03-18T23:23:25.081 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +1420us[+1431us] +/- 121ms 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -190us[ -179us] +/- 54ms 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -518us[ -507us] +/- 62ms 2024-03-18T23:23:25.082 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T23:23:25.083 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T23:23:25.093 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T23:23:25.094 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T23:23:25.108 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:23:25.203 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:23:25.276 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T23:23:25.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:23:25.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:25.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:23:25.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:25.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T23:23:25.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:25.439 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T23:23:25.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T23:23:25.469 INFO:teuthology.task.internal:Duration was 3569.305182 seconds 2024-03-18T23:23:25.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T23:23:25.485 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T23:23:25.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:23:25.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:23:25.490 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T23:23:25.529 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:23:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:23:25.533 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T23:23:25.999 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T23:23:25.999 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-03-18T23:23:26.000 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:23:26.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-18T23:23:26.020 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:23:26.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-03-18T23:23:26.044 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T23:23:26.070 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T23:23:26.070 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:23:26.073 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:23:26.087 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T23:23:26.139 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T23:23:26.139 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:23:26.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:23:26.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T23:23:26.509 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T23:23:26.522 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T23:23:26.522 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:23:26.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:23:26.577 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T23:23:26.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T23:23:26.619 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.621 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.623 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.644 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-03-18T23:23:26.646 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-18T23:23:26.670 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-03-18T23:23:26.703 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:26.718 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:26.758 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T23:23:26.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T23:23:26.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T23:23:26.786 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T23:23:26.787 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi132 2024-03-18T23:23:26.787 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:23:26.832 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi144 2024-03-18T23:23:26.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:23:26.870 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174/remote/smithi196 2024-03-18T23:23:26.871 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T23:23:26.909 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T23:23:26.909 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:23:26.912 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:23:26.914 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T23:23:26.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T23:23:26.973 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T23:23:26.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T23:23:26.989 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T23:23:26.990 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:23:26.992 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:23:26.994 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T23:23:27.004 INFO:teuthology.orchestra.run.smithi132.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:23 /home/ubuntu/cephtest 2024-03-18T23:23:27.006 INFO:teuthology.orchestra.run.smithi144.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:23 /home/ubuntu/cephtest 2024-03-18T23:23:27.010 INFO:teuthology.orchestra.run.smithi196.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 23:23 /home/ubuntu/cephtest 2024-03-18T23:23:27.012 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T23:23:27.025 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T23:23:27.059 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T23:23:27.091 DEBUG:teuthology.nuke:shortname: smithi132 2024-03-18T23:23:27.091 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:23:27.117 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.066595', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjAIo9xmZIWm7Zk56eF9gU6weQ4WnYLWAksOyYFegQ9GOc+kHgYa3cMirSiJqUKAcC6s0uwpGZLHe4uzP6aw6I='} 2024-03-18T23:23:27.121 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-18T23:23:27.121 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:23:27.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.064872', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2pEfpV+KB6JATOsaf54zwAI0UR1KbypFXMSkYc2jTtEILFQUobg01tWfQaftT5QDAeF7qG+zktm12SxLj3ZqY='} 2024-03-18T23:23:27.150 DEBUG:teuthology.nuke:shortname: smithi196 2024-03-18T23:23:27.150 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T23:23:27.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 22:15:06.062896', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgz/8U9BkJeimhJd/RJL/4J/oJQnxOuL5NVIxKukK8hBmUODGHPxMOdSalionouFaFs1RfWeW3Tcsc621V7XB8='} 2024-03-18T23:23:27.186 INFO:teuthology.orchestra.console.smithi132:Power off 2024-03-18T23:23:27.186 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:23:27.210 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-03-18T23:23:27.210 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:27.226 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-18T23:23:27.226 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:23:27.237 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-18T23:23:27.248 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-18T23:23:27.248 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:27.263 INFO:teuthology.orchestra.console.smithi196:Power off 2024-03-18T23:23:27.264 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T23:23:27.272 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-18T23:23:27.285 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-03-18T23:23:27.285 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:27.310 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-18T23:23:31.238 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:31.272 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:31.312 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:31.350 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-18T23:23:31.383 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-18T23:23:31.423 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-18T23:23:35.354 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:35.384 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:35.424 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:35.465 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-03-18T23:23:35.494 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-18T23:23:35.534 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-18T23:23:35.596 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-18T23:23:35.635 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-03-18T23:23:39.468 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T23:23:39.578 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-03-18T23:23:39.678 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-03-18T23:23:39.783 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 3569.305181503296 failure_reason: '"2024-03-18T23:01:18.036932+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-18T23:23:39.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T23:23:39.877 INFO:teuthology.run:FAIL